Liberal psychology faculty outnumber conservative psychology faculty in many higher education institutions in the United States. Despite this, new research published in Perspectives on Psychological Science indicates that the political slant of research findings (the degree to which conclusions are consistent with liberal or conservative worldviews) is not associated with the replicability of those findings. In other words, the likelihood of a study producing the same results after being repeated (or replicated) is not related to the political slant of those results.
Study author Diego A. Reinero and colleagues were interested in whether the liberal overrepresentation in psychology is influencing the “replication crisis” that psychology is currently facing, where many psychology study findings are failing to replicate. “The specific concern expressed by some critics is that a discipline composed overwhelmingly by scientists who are liberal might result in one-sided questions or mischaracterizations of other political viewpoints and that these scholars might be more lenient when reviewing liberal-leaning research (or stricter with conservative-leaning research),” the researchers wrote.
“If the research or review process was selectively compromised, it could allow the publication of liberal-leaning claims based on flimsy evidence – even if they are unlikely to hold up to scientific replication.” Reinero and colleagues call this potential unbalanced acceptance of liberal-leaning conclusions a form of liberal bias in the psychology research literature.
The researchers gathered original, published psychology studies that all had at least one replication attempt. This resulted in a sample of 194 original articles, 479 total replication attempts, and over a million participants. A group of six doctoral students (Study 1) and a group of 511 adults from the online platform Mechanical Turk (Study 2) evaluated the political slant of the original research articles by identifying the degree to which each article’s conclusions were left-leaning/consistent with a liberal worldview or right-leaning/consistent with a conservative worldview.
Overall, the original research findings were rated to be only slightly more liberal than conservative. After analyzing the ratings from both Studies 1 and 2, the researchers did not find any evidence for the relationship between political slant and likelihood of replication. Similarly, political slant was not associated with factors that typically inform statistical robustness such as sample size and the magnitude of the research findings (effect size). In other words, political slant did not affect the overall statistical quality of the research. The researchers also found no relationship between political slant and the number of times an original article has been cited.
Altogether, this research shows that the best predictor of the replicability of a study is the robustness of the original study’s statistics, not the political slant of the findings. “Liberal findings were just as likely to be replicable and, in exploratory analyses, were as statistically robust as conservative findings and as likely to be cited or mentioned in the media,” wrote Reinero and colleagues.
Although these results might be reassuring against a liberal bias in the replication crisis, the researchers addressed some limitations of this study. The authors mentioned their sample was “limited to studies for which replication data were readily available and thus was not completely representative of the entire field.” Another potential caveat is the narrow definition of political slant. “Labels such as ‘liberal’ and ‘conservative’ may be too broad to capture the nuanced ideologies and assorted political attitudes of people,” wrote the researchers.
“Our data suggest that the political skew of psychologists is not tightly coupled with the political skew of the literature itself, and future work should seek to disentangle these discrepancies,” the researchers said.
The study, “Is the Political Slant of Psychology Research Related to Scientific Replicability?“, was authored by Diego A. Reinero, Julian A. Wills, William J. Brady,
Peter Mende-Siedlecki, Jarret T. Crawford, and Jay J. Van Bavel.