The problem is many of them take it as a personal attack, as evident by the comments here, and they can't even see how that is the problem.
I'm really proud of some of the comments like yours, danilocampos', and msbarnett's. It's just a shame you're being met with the same crap most of these conversations have in the programming community.
Of course it is an attack. A conference was shut down. If in the future all conferences have to make an effort for diversity, it is a tax on all of us - so it is an attack. If somebody gets my job because they are female or black, it is an attack. If I get blamed for not making an extra inclusive effort, it is an attack.
Actually I think what makes me angry is that notion that white men have handed everything to them on a plate. That is very far from reality, and dismisses a lot of human experience. Life is much more complicated than the height of your paycheck.
This isn't a personal attack on you and your lifestyle and your people. It's a call to broaden the church.