Researchers from various fields have long been fascinated by the human capacity for theory of mind – our ability to understand and predict the mental states of others. This capacity underpins much of our social interactions, from interpreting indirect requests to detecting deception.
Recently, a study published in Nature Human Behaviour revealed that advanced artificial intelligence (AI) models, particularly large language models like OpenAI’s GPT-4, demonstrate notable competence in performing tasks designed to test theory of mind. GPT-4 often matched or even surpassed human performance in understanding indirect requests, false beliefs, and misdirection, but it struggled with detecting faux pas.
Large language models (LLMs) are built using deep learning techniques and trained on vast amounts of text data. They function by predicting the next word in a sequence, allowing them to generate coherent and contextually appropriate text based on the input they receive.
The training process involves exposing the model to diverse linguistic patterns, enabling it to learn grammar, facts about the world, and even some elements of reasoning and inference. LLMs have shown remarkable capabilities in various tasks, including language translation, summarization, and conversation, making them powerful tools for a wide range of applications.
Despite their impressive performance, LLMs are not without limitations. Their ability to mimic human language has led to questions about whether they truly understand the content they generate or if they are simply regurgitating patterns learned during training. This distinction is particularly important when considering tasks that require a deep understanding of context and human psychology, such as those involving theory of mind.
“Theory of Mind is an important aspect of human cognition as it allows us to navigate our social environment easily and efficiently by tracking the mental states of people around us,” explained study author James Strachan, a Humboldt Research Fellow at the University Medical Center Hamburg-Eppendorf.
“Given the importance of this capacity for social interactions among humans, this is also a key consideration in the ongoing development of AI technologies that aim to allow for fluent human-AI interactions. Evaluating how well AIs (such as the LLMs we tested) can engage in mentalistic inference (that is, drawing conclusions about people’s mental states from their behavior) requires a systematic approach with comparison against human samples.”
To rigorously evaluate the theory of mind capabilities of large language models (LLMs), the researchers designed a study involving various tasks that test different aspects of this cognitive ability. The study primarily focused on comparing the performance of GPT-4, its predecessor GPT-3.5, and another language model known as LLaMA2-70B against human participants.
The researchers selected a battery of well-established psychological tests that are typically used to assess theory of mind in humans. These tests included false belief tasks, irony comprehension, faux pas detection, hinting tasks, and strange stories.
The human participants were recruited online through the Prolific platform, ensuring they were native English speakers aged between 18 and 70 years, with no history of psychiatric conditions or dyslexia. A total of 1,907 participants were involved in the study, with specific numbers allocated to each test. Each AI model was tested across 15 independent sessions per task, ensuring that each session simulated a naive participant, as the models do not retain memory across different chat sessions.
The researchers found that GPT-4 demonstrated impressive theory of mind capabilities, matching or exceeding human performance in some tasks.
“It is surprising that these models can engage in such sophisticated social reasoning without the direct embodied experience that typifies human development,” Strachan told PsyPost. “The fact that these models, which are trained extensively on the statistics of natural language through the use of large data sets, can solve these tasks indicates that a lot of how we think and reason about others is encoded in the language that we use and, excitingly, can be reconstructed in part from the structure of this language.”
False Belief Tasks
In the false belief tasks, participants were presented with scenarios where a character’s belief about the world differed from reality. Both GPT-4 and GPT-3.5 performed at ceiling levels, correctly predicting where a character would look for an object based on their false belief, much like human participants. This task measures the ability to inhibit one’s own knowledge and predict others’ actions based on their mental states, a fundamental aspect of theory of mind.
Irony Comprehension
Irony comprehension required participants to interpret statements where the intended meaning was the opposite of the literal meaning. GPT-4 excelled in this task, outperforming human participants by accurately identifying ironic remarks more frequently. This reflects an advanced understanding of non-literal language and suggests that GPT-4 can grasp the subtleties of social communication. In contrast, GPT-3.5 and LLaMA2-70B showed more variability, with LLaMA2-70B struggling significantly in distinguishing ironic from non-ironic statements.
Faux Pas Detection
The faux pas detection task involved identifying when a character said something inappropriate without realizing it. Here, GPT-4 struggled notably. While it could recognize that a statement might be hurtful, it often failed to correctly identify that the speaker was unaware of the context that made their statement inappropriate. GPT-3.5 performed even worse, almost at floor levels, except for one item. Interestingly, LLaMA2-70B outperformed humans in this task, correctly identifying faux pas in nearly all instances. This suggests that while GPT-4 has strong capabilities in some areas, it has notable weaknesses in integrating context to infer ignorance.
Hinting Tasks
In hinting tasks, participants had to interpret indirect speech to understand implied requests. GPT-4 again showed superior performance, better than humans, in identifying the intended meaning behind hints. This demonstrates the model’s strong ability to infer intentions from indirect language. GPT-3.5’s performance was on par with humans, while LLaMA2-70B performed significantly below human levels.
Strange Stories
The strange stories task involved explaining characters’ behaviors in complex social scenarios, requiring advanced reasoning about mental states. GPT-4 performed exceptionally well, significantly better than humans, in explaining the characters’ actions and intentions. This indicates that GPT-4 can effectively navigate complex social narratives. GPT-3.5’s performance was comparable to humans, while LLaMA2-70B scored lower, indicating challenges in handling more sophisticated social reasoning.
Despite the promising results, the study highlighted several limitations. One significant issue is the potential for AI models to rely on shallow heuristics rather than robust understanding. For example, GPT-4’s failure in faux pas detection might stem from an overly cautious approach, termed “hyperconservatism,” where the model avoids committing to an explanation when context is ambiguous. This behavior could be influenced by mitigation measures designed to reduce the generation of inaccurate or inappropriate responses.
To further investigate, the researchers conducted follow-up experiments. They rephrased the faux pas questions in terms of likelihood, asking whether it was more likely that the speaker knew or did not know the context. This change led to perfect performance from GPT-4, supporting the hypothesis that the model’s initial failures were due to caution rather than an inability to make inferences.
“Our study demonstrates that LLMs, particularly GPT-4, demonstrate good competence at solving tasks aimed at testing Theory of Mind in humans,” Strachan said. “While their responses differ from humans in some ways (such as GPT-4’s conservatism in identifying faux pas), they are able to demonstrate sensitivity to the mental states of humans in third-person stories that resembles that of high-performing humans.”
The findings suggest that while GPT-4 and similar models exhibit impressive capabilities in theory of mind tasks, there are distinct differences in how they process and respond to social information compared to humans.
“As the tests we used were designed and validated for use with humans to test the function (or dysfunction) of Theory of Mind, they rely on certain assumptions that are fair of human subjects but are inappropriate to make of LLMs (e.g. that the subject has a mind in the first place),” Strachan said.
“We do not want to imply that there is a resemblance between LLMs and the underlying social reasoning processes that human minds are capable of; our study only measured the performance of humans and LLMs, and is not suited to forming deep conclusions about the nature of any cognition-analogous processes in machines.
“Even if we wanted to say this, Theory of Mind is much more than being able to answer test questions in isolated conditions; we have no evidence yet that LLMs would be capable of using their capacity to make mentalistic inferences in order to guide their interactions with people as humans do.”
Regarding future research, Strachan remarked that “we have a few avenues we would like to pursue, one of which is to study the limits of these capacities in naturalistic interactions and how the appearance of Theory of Mind in AI interaction partners affects the behavior and judgements of human users.”
The study, “Testing theory of mind in large language models and humans,” was authored by James W. A. Strachan, Dalila Albergo, Giulia Borghini, Oriana Pansardi, Eugenio Scaliti, Saurabh Gupta, Krati Saxena, Alessandro Rufo, Stefano Panzeri, Guido Manzi, Michael S. A. Graziano, and Cristina Becchio.
