I totally agree with you, except I don't think fear is quite the right way to frame it, and perhaps more that many women know they will be talked down to, make a rational decision that they don't want to put up with that, and do something else with their time.
Afraid doesn't necessarily denote the emotion of fear, I think in this context we can safely say it's merely the anticipation of something which is disliked. Like, "I'm afraid if I order from Sloppy Joe's Pizzeria, I'll find another hair in my pizza."