It's not the infrastructure that he cares about. He believe that any provider you use should be free to run their infrastructure as they see fit, releasing or not releasing any code as they see fit. It's the fact that the software that you run, on your end, is non-free.
With Skype, you cannot connect your own client to their network; you must run their proprietary, highly secretive client.
With the phone system, you can attach any device that you want. You can use a dumb, analog phone, or a digital phone running free software, or a digital phone running proprietary software, or modem, or a fax machine, or whatever, to the phone system.
Now, he also does have other concerns about the infrastructure; he would generally prefer to support infrastructure running on open tools and technologies, and may have various other concerns with particular providers.
But he sees that as a separate question of the ethical question about running non-free software on his own computers.
With Skype, you cannot connect your own client to their network; you must run their proprietary, highly secretive client.
With the phone system, you can attach any device that you want. You can use a dumb, analog phone, or a digital phone running free software, or a digital phone running proprietary software, or modem, or a fax machine, or whatever, to the phone system.
Note that this was not always the case (https://en.wikipedia.org/wiki/Carterfone, https://en.wikipedia.org/wiki/Hush-A-Phone_v._United_States).