New research published in the Journal of Applied Psychology casts doubt on the idea that awareness of negative stereotypes substantially and systematically impair the performance of certain groups.
“I was originally interested in stereotype threat because of its potential effects on test-takers. My advisor and I decided to start this meta-analysis after two individual studies on the topic were retracted due to data fabrication,” said study author Oren Shewach, a research scientist at the Human Resources Research Organization.
“While these retracted studies showed very large effects, we found that they had minimal impact on the overall threat effect. As I became more familiar with this research, I realized that some of the experimental conditions used were dissimilar from high stakes testing settings. This shaped the current focus of the paper, which is on stereotype threat in high stakes testing settings.”
The researchers conducted a meta-analysis of 212 experimental studies on stereotype threat, which included a total of more than 10,000 adult participants — making it the largest meta-analysis of stereotype threat to date.
“Stereotype threat is the situation where someone is concerned about being treated negatively on the basis of a stereotype about their group. Stereotype threat has long been thought to influence people on tests, including high-stakes tests (i.e., college admissions and employment tests),” Shewach explained.
Some studies used blatant invocations of stereotype threat, such as by telling female participants that women usually perform worse than men on the test they’re about to take. Other studies introduced stereotype threat in more subtle ways.
Unsurprisingly, the researchers found that blatant manipulations of stereotype threat tended to produce stronger effects than subtle manipulations. But such blatant statements are rarely encountered before real-world tests.
“When we restrict stereotype threat research to studies showing similarity to high stakes testing-like conditions, the stereotype threat effect decreases substantially. Overall, our results indicated that the size of the threat effect experienced in adults on high stakes cognitive ability tests may range from negligible to small,” Shewach told PsyPost.
“This conclusion is supported by evidence from: (a) studies with lab conditions similar to high stakes testing settings, (b) a small sample of actual high stakes tests, (c) studies that incentivize participants, financially or otherwise, and (d) by tests of publication bias.”
The first study on stereotype threat was published in 1995. Since then, the phenomenon has frequently been used to explain why some groups tend to score higher on cognitive tests than others.
The new findings cast doubt on that explanation. However, the results do not mean that stereotype threat does not exist at all.
“We do not question the existence of the stereotype threat phenomenon; we just question the evidence that it affects cognitive ability tests in high stakes settings in a systematic way. Also, individual test-takers may still be affected by stereotype threat. However, we are looking at systematic effects, and average effects across groups,” Shewach explained.
“Fewer than 10% of studies in our sample contained a motivational incentive that might serve to increase test-taker motivation beyond a minimal level. It would still be useful to conduct threat research in a large-scale sample with motivational incentives, also incorporating other features similar to high stakes testing settings.”
The study, “Stereotype Threat Effects in Settings With Features Likely Versus Unlikely in Operational Test Settings: A Meta-Analysis“, was authored by Oren R. Shewach, Paul R. Sackett, and Sander Quint.