A recent study employed an innovative method to examine whether conservatives or liberals are better at discerning between true and fake news stories. The findings, published in Public Opinion Quarterly, suggests that both conservatives and liberals have their own weaknesses when it comes to evaluating media information. Conservatives were more likely to struggle with identifying false information, while liberals were more likely to trust information that supported their own beliefs, even if it wasn’t accurate.
Understanding the differences between liberals and conservatives in their behavior and thinking has been a topic of interest in political science for a long time. Previous studies have looked at different aspects like political tolerance, rigid thinking, knowledge, overconfidence, and trust to see if there are any variations between liberals and conservatives.
One area of particular interest is how liberals and conservatives process news, especially with the rise of “fake news” during the 2016 U.S. elections. However, previous studies on this topic have given mixed results. Some found differences between liberals and conservatives, while others found no significant differences.
“This is of course a controversial topic in the public discourse: Is one ideological side ‘worse’ at processing information than the other?” said study author Bernhard Clemm von Hohenberg, a research associate at GESIS Institute for the Social Sciences in Cologne.”You can see hot takes on the question everywhere – often thinly veiled attempts to dunk on the other side.”
“At the same time, the scientific evidence on the question is very mixed. I felt that one reason for the mixed evidence was that typical research designs are sloppy, as they chose arbitrary sets of news to confront their study participants with. So my hope was to contribute to research about left-right differences with an innovative, hopefully better method.”
The researcher selected news items by sampling from a collection of political news. Since there is no comprehensive list of all sources that publish political news, the researcher used the Google News archive and fact-checkers’ archives as approximations. This included both true and false information.
The researcher collected news items from the Google News API endpoint, focusing on items from US sources classified as “general,” “business,” “health,” and “science.” This resulted in a collection of 775,282 items. To complement the data from Google News, the researcher used fact-checkers’ archives from “Snopes,” “PolitiFact,” and “Truth or Fiction.” Only items clearly labeled as completely true or completely false were included, resulting in 710 items.
To ensure the sample represented the larger population of news items, an iterative procedure was used. Initially, a random sample of 40 items was selected from each collection, excluding items that didn’t meet the target population criteria. This process was repeated until two samples of 40 items each were obtained.
Out of the fact-checked items, 9 were labeled as true and 31 were labeled as false. The researcher personally fact-checked the items from Google News and found all of them to be true. So, out of the total 80 news items, 31 were false and 49 were true.
“It is actually very hard to answer the question whether conservatives and liberals are ‘better’ at processing information,” Clemm told PsyPost. “The reason is that with so much information out there in the real world, it is not so clear which information to ask our study participants about! So when reading about studies trying to answer this question, ask yourself how they selected the information.”
The ideological bias of the news items was measured using 385 raters from MTurk and Prolific. Participants rated the items based on whether they were more favorable to liberals or conservatives and whether they were more consistent with the beliefs of liberals or conservatives. The results showed that false items had a slightly wider range of ideological bias compared to true items, and false items were more often in line with the beliefs of conservatives than liberals.
For the main survey, 1,393 participants from the U.S. were recruited through the survey company Dynata. The sample was designed to be representative of the U.S. population in terms of gender, age, education, and political affiliation.
In the survey, participants indicated their position on a seven-point ideology scale and provided information on various factors like political affiliation, Facebook usage, trust in the media, digital literacy, economic and social policy attitudes, cognitive reflection, ethnicity, state of residence, and income.
Participants were then presented with eight randomly selected news items from the previously collected set of 80 news items. After reading each item’s text and headline, participants rated their belief in the accuracy of the information. Unlike in previous research, the participants were informed that the information could be true or false.
The results of the study showed that conservatives were less influenced by the truth value of news items when forming their beliefs compared to liberals. Conservatives showed a smaller difference in belief between true and false news items. On the other hand, liberals showed a greater difference in belief based on how well news items aligned with their ideological preferences compared to conservatives. Conservatives, however, were less influenced by the alignment of news items with their beliefs.
“Although conservatives were worse at discerning false from true, it was interesting that liberals were more inclined to believe information in line with their ideology – whether this information was true or false! In other words, in my study, both sides on the political spectrum showed some flaws in information processing,” Clemm told PsyPost.
The study also examined other factors, such as cognitive reflection and education, that could explain these differences but found that the asymmetry between conservatives and liberals in perceiving truth and bias persisted. The fact that the differences persisted even after considering these potential confounding variables suggests that political ideology has a unique influence on how conservatives and liberals evaluate news items.
While the study provides valuable insights into the differences between conservatives and liberals in their ability to discern true from false information and their susceptibility to bias, it is important to acknowledge its limitations. For example, the study’s sample of news items was collected from Google News and fact-checkers’ archives. While efforts were made to ensure representativeness, the selection process may still introduce some biases. Additionally, the study focused on U.S. sources, which may limit the generalizability of the findings to other countries or contexts.
“I went to great lengths to collect a representative sample of real-world information,” Clemm said. “But of course my sample has its own problems, for example its reliance on fact-checkers, who are likely to have their own biases in selecting content. Good samples of information are really hard to get. We need future studies to explore more and better ways!”
The study, titled “Truth and Bias, Left and Right: Testing Ideological Asymmetries with a Realistic News Supply,” was published April 29, 2023.
