I would agree that there is an artfulness, for lack of a better term, in engineering. However, there is a lot of science and math i.e. structural engineering that underlies building a structure that is safe and won’t fall down. Admittedly it still does happen from time to time due to mistakes or things that might have been misunderstood, e.g. the Tacoma Narrows Bridge.
The field of software engineering is very nascent and is still in need of the underlying science and math foundations like you find in other engineering disciplines. Unfortunately methodologies like Agile and Waterfall seem to have become synonymous with software engineering.
Right, but when you hire an architect to design a random office building, you're not expecting a piece of art. My point is the vast majority of programming projects are random office building # 23.
Yes, and we should be encouraging more of that. But as programmers (well to my mind anyway) it nags us when we look at inefficient solutions, even those which are reasonably robost/adequate. We tend to scoff at people using Visual Basic to solve a business problem, when its probably the best language for a large chunk of the problem space.
Civil architects can be wildly creative and artful in their industry which understands its own domain deeply.