But the authors do point out what are intrinsic biases and failures of experimental design in most of the examples they mention
* Inferring sexual orientation: Linking «self-reported sexual orientation labels» with «[...]scraped their data from social media profiles, claiming that training their classifiers on “self-taken, easily accessible digital facial images increases the ecological validity of our results.”[...]». Social media profile photos are by their very nature socially influenced, with open sexual orientation being an important cue to display.
* Personality psychology: Training and test datasets came from the same pool of «participants [who] self-reported personality characteristics by completing an online questionnaire and then uploaded several photographs». This heavily suggests that the participants were aware when choosing the photos that this was a "personality type" experiment, and may even have made their own awareness of their personality more salient by doing the test first and then uploading the photographs.
* “Abnormality” classification: General critique of lack of transparency as to how the true labels were determined.
* Lie detection: The ability to detect the facial differences between people following two different experimental instructions does not equate to lie detection.
* Criminality detection: At least they used official ID photographs instead of self-selection-biased photos like the first example... but consider this: what conclusions would their same model reach if it used official ID photos of US populations? The confounding factors of class and ethnicity are obvious.
These are examples and the experimental designs were a particular choice by the authors of those examples - they aren't intrinsic to ML or the ML community.
Hence why the authors never claim to be talking about ML or the ML community. They are talking about "the harmful repercussions of ML-laundered junk science" and, in the section that I quoted in my comment above, they "review the details of several representative examples of physiognomic ML".
* Inferring sexual orientation: Linking «self-reported sexual orientation labels» with «[...]scraped their data from social media profiles, claiming that training their classifiers on “self-taken, easily accessible digital facial images increases the ecological validity of our results.”[...]». Social media profile photos are by their very nature socially influenced, with open sexual orientation being an important cue to display.
* Personality psychology: Training and test datasets came from the same pool of «participants [who] self-reported personality characteristics by completing an online questionnaire and then uploaded several photographs». This heavily suggests that the participants were aware when choosing the photos that this was a "personality type" experiment, and may even have made their own awareness of their personality more salient by doing the test first and then uploading the photographs.
* “Abnormality” classification: General critique of lack of transparency as to how the true labels were determined.
* Lie detection: The ability to detect the facial differences between people following two different experimental instructions does not equate to lie detection.
* Criminality detection: At least they used official ID photographs instead of self-selection-biased photos like the first example... but consider this: what conclusions would their same model reach if it used official ID photos of US populations? The confounding factors of class and ethnicity are obvious.