A similar issue arises with health insurance. Using AI to evaluate claims is a huge efficiency play and you don’t have much ability to fight it if something goes wrong. And even if you can these decisions can be life or death in the short term and human intervention usually takes time.
I'm not entirely sure what you're getting at re: hype.
While there is undoubtedly a lot of hype around these tools right now, that hype is based on a pretty major leap in technology that has fundamentally altered the landscape going forward. There are some great use cases that legitimize some of the hype.
As far as concrete examples, see the sibling comment with the anecdote regarding health insurance denial. There are also portions of the tech industry focused on rolling these tools out in business environments. They're publicly reporting their earnings, and discussing the role AI is playing in major business deals.
Look at players like Salesforce, ServiceNow, Atlassian, etc. They're all rapidly rolling out various AI capabilities into their existing customer bases. They have giant sales forces all actively pushing these capabilities. They also sell to governments. Hype or not, it adds up to real world outcomes.
Public statements by Musk about his intention to use AI also come to mind, and he's repeatedly shown a willingness to break things in the pursuit of his goals.
There are already active discussions about AI being used in government for “efficiency” reasons.