Silly remark. The trolley problem involves the unavoidable death of at least one party - the decider is not among those who die. Not analogous.
The Uber that killed Elaine Herzberg in 2018 failed to solve the only problem it had. Did the CPU die? Did Uber or Uber's president?
My decision saved the 3 lives in the oncoming vehicles (including the idiot who decided to try passing the gasoline tanker in their rattletrap), as well as my own and those of the two additional children I eventually had (and their children).
I was going to say, the vast majority of weird near crash situations can be dealt with by slowing/stopping. That is probably not so hard to program in and not so different from what humans do.
As of a few years back, the 'FSD' Tesla had failed to detect several parked police cars, a parked firetruck, a moving semi pulling in front of it, and a concrete lane divider, all of which it struck without slowing down at all. At least two dead operators.
You're describing a utopia where outcomes are perfectly predictable, which is silly.
"And roads should be without any defect, weather should never happen, pedestrians and other drivers should always behave rationally, and black ice is not legally allowed to exist."
Exactly. The real world that real people have to drive in is far, far more complex than these sloppy machines could ever know. The map is not the territory.
The car should be traveling at a speed where it can break and stay in its lane to avoid any at fault accident. Nothing more, nothing less.