By that logic the vast majority of users of whatsapp, signal and most other e2ee protocols/apps use it in a useless way, right? Most people I know who use these apps (even the security-conscious ones) never verified the key.
Signal tells you outright when someone's key has changed, though. It's usually pretty trivial to establish that the original conversation is authentic when you're just talking with people you know in real life (where an impersonation attempt would likely fail for numerous reasons), and you can assume that device is still theirs until their key changes.
There is still a risk that someone is running a MITM attack. The initial conversation would be authentic, but the key belongs to someone else who is just forwarding the messages. Your communications would no longer be private and they could switch from passive eavesdropping to impersonation at any point without changing the key.
Most people rotate their Signals keys every time they rotate their phone hardware (which is inexplicably often for some people apparently), because keys are just auto-accepted everywhere so there is no real incentive to bother moving them. In larger groups there's always someone.
It isn't helped by the fact that the backup process is a bit obscure and doesn't work cross operating systems. For the select few that cares, verifying keys is effective against attackers who aren't Signal themselves, Google or in control of the Play Store. Just make sure to keep an eye out for that key changed warning, it's easy to miss.
