There's probably little danger to the savvy user who understands how manipulative technology like this can be.
The problems come when vulnerable users are targeted using dark patterns. How AI dark patterns will evolve is very uncertain [1] however I suspect they will be extremely subtle and very effective.
What's the worst that can happen if someone vulnerable is persuaded to buy a flight by an AI. I don't know, maybe depression and bad credit after the chatbot's promises weren't met. If they're persuaded to buy a weapon, that's a different matter.
At least current advertising is somewhat public, although that's increasingly less true as ads get more targeted.
This is new territory where ads will be so extremely private it will be only known by the user (maybe they won't even notice) and someone reading the subpoenaed chat logs after a user does something terrible. Those chat logs will likely be inconclusive anyway.
The problems come when vulnerable users are targeted using dark patterns. How AI dark patterns will evolve is very uncertain [1] however I suspect they will be extremely subtle and very effective.
What's the worst that can happen if someone vulnerable is persuaded to buy a flight by an AI. I don't know, maybe depression and bad credit after the chatbot's promises weren't met. If they're persuaded to buy a weapon, that's a different matter.
At least current advertising is somewhat public, although that's increasingly less true as ads get more targeted.
This is new territory where ads will be so extremely private it will be only known by the user (maybe they won't even notice) and someone reading the subpoenaed chat logs after a user does something terrible. Those chat logs will likely be inconclusive anyway.
[1] https://venturebeat.com/ai/darkness-rising-the-hidden-danger...