That's faulty. Solar installations change the albedo of the planet. That's not a problem if the fraction is a very small one and it is spread out. But if it is concentrated or relatively large it certainly could have an effect.
We thought much the same when we introduced the automobile. Plastic for packaging purposes. Freon to help with the Carnot cycle in refrigerators. These things don't matter when you do them once or twice. But when you start doing them on an industrial scale it changes the equation from 'no effect' to 'unknown effect'. And unknown effect might be anything from negligible to planet wide catastrophe. It would be nice to know where we land before taking off.
Oceans warming up is a big thing, and local effects can be substantial even if global average change is negligible.
Yes, an increase in reflectivity would cool the planet down. But solar panels actually absorb a lot more than they reflect (they would have to). The ideal solar panel would be utterly black.