I did a few internships in college with the State Department, in an agency similar to the one mentioned in the article. You describe it very accurately, and summarize my primary reason for not returning full-time after I graduated. In my opinion, the clearance process selects for three things: 1) those who are extremely rule-abiding -- often, as you describe, the "goody-two-shoes" from school. For example, most of these people have never touched drugs (since the clearance process involves polygraphs on drug use, as well as constant drug checks), and they often have strong disdain for those who have. These are not qualities I think one should have when they are dealing with diverse types of people and organizations. Someone in this position should have knowledge and nuance when it comes to drugs, even if they personally have never used them. Also, they definitely select for people who are exactly the "tattle tales" you allude to. I won't get into the details of the sort of questions they ask that select for such things, but I'm sure you can use your imagination. 2) People who are not receiving offers elsewhere in industry. The clearance process takes forever, sometimes can be up to three years to get a clearance. For college students, they would "expedite" it to take 6 months, but regardless -- most people with any marketable skills get snatched up by a better-paying, faster moving organization. 3) Most importantly, I think it selects for the type of egotistical person you are describing. Many of the people I encountered during my time there were very macho, patronizing types, who think they've "seen it all". To be sure, some of them have been through a lot of difficult things, and I don't mean to downplay the things they've seen. But there are probably a good number who haven't, and just like to act as though they're really tough. My theory on this is that people with smaller egos get more nervous during the polygraph process and are thus more likely to fail (since nerves are often the reason people appear to be lying, due to fast heart-rate, twitches, etc).

This seems like a recipe for disaster.

I read "The Wise Men" and was under the impression that the State department was staffed with incredibly smart folks who served out of a sense of duty to the Country rather than as a job of last resort.

Makes me wonder: how do you think the State Department would have changed under the leadership of Tillerson and now Pompeo?

It wouldn't have. The culture of the organization (and by proxy the culture across all of those organizations -- including allied/FiveEyes nations) is seen as one of its greatest assets, and they go to great lengths to preserve that culture and boot outsiders. If I hadn't been part of a completely third party organization, there's no way I'd have been in those positions for as long as I was ("those positions" here not implying I performed the same function, but rather a job that was critical to their continued presence abroad, with my clearance under the control of that external third party -- sorry for the lack of specificity). No leader will have been able to change their culture, and any leader making headway into changing the culture will not have lasted very long.