When people debate highly contentious issues, they often misrepresent the other side’s arguments. A recent study tested whether individuals can accurately describe the beliefs of their political or social opponents, finding that people generally perform quite well at understanding views they disagree with. These results were published in Cognitive Science.
Measuring open-mindedness has historically been a challenge for psychologists. Traditionally, researchers use self-report surveys that ask people to rate their own intellectual habits. A participant might be asked to agree or disagree with statements about whether they listen to contradictory evidence. Other questionnaires ask individuals to indicate if they believe changing their mind is a sign of weakness.
While these surveys are easy to administer, they have known methodological flaws. People often answer in ways that make themselves look good to the researchers. This tendency is known in psychology as social desirability bias. Individuals might also lack the ability to accurately view their own thought processes, leading them to claim objectivity even when they routinely ignore contradicting evidence.
To bypass the problems with self-reporting, researchers sought a behavioral measurement. They designed a task inspired by a classic concept in cognitive science known as the Turing Test. Originally proposed by mathematician Alan Turing in the mid-twentieth century, the basic Turing Test evaluates whether an artificial intelligence can successfully mimic a human. An algorithm passes the test if a human judge cannot tell whether they are conversing with a machine or another person.
An economist named Bryan Caplan previously proposed adapting this concept to human debates. In an Ideological Turing Test, a person must argue an opposing viewpoint well enough to convince someone who actually holds that perspective. The original idea suggests that if you can seamlessly pass as your opponent, you truly comprehend their worldview.
Psychologists Charlotte O. Brand, Daniel Brady, and Tom Stafford at the University of Sheffield put this concept into actual practice. They wanted to see if an experimental design could reliably document open-mindedness without relying on introspection. The team chose three polarizing topics in the United Kingdom to serve as the foundation of the study. These topics were veganism, the COVID-19 vaccines, and the Brexit vote to leave the European Union.
They recruited 600 participants from across the country. Each person held a strong opinion on at least one of these three issues. In the first phase of the experiment, participants provided arguments supporting their own beliefs. A vegan, for example, was asked to imagine chatting with a fellow vegan and to write out three reasons that person might give for following the diet.
In the second phase, the researchers asked these same individuals to switch perspectives entirely. The vegan participant had to imagine chatting with a meat-eater and write out three reasons that person might give for avoiding a vegan diet. This process generated thousands of genuine and mimicked arguments.
The researchers processed these submitted arguments to prepare them for a blind review process. They converted all the statements into the first person to make them sound like genuine beliefs. For example, an argument originally written as “they might be scared of needles” became “I am scared of needles.”
The researchers also scrubbed the text of any sarcastic statements. Derogatory comments like “they think the vaccine will track them” were changed to “I think the vaccine will track me.” This preparation ensured that the arguments appeared earnest and did not give away the true identity of the writer.
Next, a new group of 1,200 participants took on the role of judges. These raters read the prepared arguments and indicated how strongly they agreed with each statement on a seven-point scale. The judges did not know if the arguments were written by genuine supporters of an issue or by opponents masquerading as supporters.
The research team established a baseline threshold for passing the test. They calculated the average agreement score that genuine supporters gave to arguments written by their fellow supporters. An argument written by an opponent passed the test if it scored at or above this baseline average.
This relative baseline accounted for differences in how much certain groups generally agreed with their own side. Vegans, for example, rated pro-vegan arguments highly across the board. Non-vegans tended to remain ambivalent regarding both perspectives. Establishing a relative metric prevented misleading success rates in the overall data.
The results revealed a high baseline of mutual understanding among the participants. People across all three debates were generally competent at mimicking the opposing perspective. The success rates reliably varied depending on the issue in question.
About 54 percent of the arguments meant to mimic the opposing side on the topic of COVID-19 vaccines passed the test. The arguments regarding the Brexit vote saw a 64 percent pass rate. Mimicked arguments about veganism reached the highest success metric, with a 71 percent pass rate.
The researchers observed no evidence that either side of a specific debate was inherently better at passing the test. People who voted to leave the European Union were just as capable of predicting remainer arguments as remainers were at predicting leaver arguments. Being part of a minority group, like a vaccine refuser, did not predict better performance.
Passing the test offered predictive insights into how people viewed their ideological rivals. The Sheffield team checked their behavioral results against standard questions about opponent perception. Those who successfully mimicked the other side were less likely to judge their opponents negatively.
Specifically, successful participants were less likely to rate people with opposing views as ignorant or irrational. They were also less likely to perceive their rivals as immoral or unethical. Additionally, individuals who passed the test were more likely to concede that their opponents had good arguments for their beliefs.
The team recorded how much time participants self-reported researching their chosen topics or discussing them with opponents. They predicted that high engagement and frequent debate would correspond with better test performance. This hypothesis failed to materialize in the collected data.
Self-reporting frequent discussions with opposing groups did not predict a higher passing rate. In the specific case of veganism, individuals who reported spending the most time researching the topic were actually less likely to pass certain metrics of the test. High engagement with a topic does not automatically correlate with better perspective-taking. Spending extensive time reading about an issue might actually isolate a person from the actual views of the opposing side.
The researchers note that open-mindedness might not be a stable personality trait that remains constant over a lifetime. It seems to respond to specific situations and personal environments. A person might be highly capable of understanding an opponent in one domain but struggle to do so regarding a different issue.
Future work might explore how the ability to articulate opposing reasons relates to broader cognitive abilities. Psychologists study a phenomenon known as theory of mind, which involves the capacity to attribute mental states and distinct beliefs to other people. Investigating how natural differences in theory of mind predict debate performance could provide new insights into human social behavior.
Overall, the study provides a new behavioral tool for the scientific community. By moving away from surveys that rely on introspection, researchers can accurately document how we interact with differing opinions. This design could eventually help gauge whether various conflict resolution programs actually improve our ability to understand each other.
The study, “The Ideological Turing Test: A Behavioral Measure of Open-Mindedness and Perspective-Taking,” was authored by Charlotte O. Brand, Daniel Brady, and Tom Stafford.