I don't think he was talking about verifying the software itself. Even if the software was open source, would it have been formally verified to make sure it has zero bugs? It is better to check the calculations by hand to make sure that no critical errors make it into the final design.
If you are checking the calculations by hand, then it seems to me you are verifying the software. It's not a bad thing to do, but I wonder why it's considered acceptable when the results don't match your own? If your results are correct, then you have a malfunctioning tool...
Wouldn't it be helpful to be able to look at the code to see what it is actually doing behind the scenes?
Not if you can't code, which most engineers cannot (nothing wrong with that). In fact, I'd say without a team of highly trained software engineers and a whole lot of time and money, having the code for a CAD program is almost useless. In civil engineering, it's my understanding that it's standard procedure to check everything CAD tools output. If a bug leads to failure in the field, at least one person, but probably multiple people, has failed horribly and he will rightfully be held responsible.
If the whole point if verification is that you don't trust your software, then that's not a very good idea. So you use the ultimate slow-but-reliable computer, the trained engineer.