Subscribe
The latest psychology and neuroscience discoveries.
My Account
  • Mental Health
  • Social Psychology
  • Cognitive Science
  • Psychopharmacology
  • Neuroscience
  • About
No Result
View All Result
PsyPost
PsyPost
No Result
View All Result
Home Exclusive Artificial Intelligence

ChatGPT-4 outperforms human psychologists in test of social intelligence, study finds

by Eric W. Dolan
April 19, 2024
in Artificial Intelligence
(Photo credit: Adobe Stock)

(Photo credit: Adobe Stock)

Share on TwitterShare on Facebook
Stay informed on the latest psychology and neuroscience research—follow PsyPost on LinkedIn for daily updates and insights.

A new study published in Frontiers in Psychology investigates how AI compares to human psychologists in understanding and responding to human emotions and needs during counseling. The study specifically examined large language models, such as ChatGPT-4, Google Bard, and Bing, assessing their social intelligence — a critical skill in psychotherapy.

ChatGPT-4 outperformed all participating psychologists, while Bing surpassed more than half of them. However, Google Bard’s performance was comparable only to psychologists seeking bachelor’s degrees and was significantly outstripped by doctoral students.

Large language models (LLMs) are advanced artificial intelligence systems designed to understand and generate human-like text by processing vast amounts of written data. These models are trained on diverse internet text to capture nuances in language, context, and syntax.

Through techniques known as deep learning, particularly using structures called neural networks, LLMs can perform a variety of tasks such as answering questions, translating languages, summarizing long articles, and even engaging in conversation that feels strikingly human.

While previous research has shown that LLMs can diagnose and help manage mental health conditions, there was a gap in understanding specifically how these models perform in social contexts, particularly against human psychologists who are skilled in navigating complex emotional interactions.

“The use of artificial intelligence models in counseling and psychotherapy represents a major challenge for psychologists, due to concern that it may take their place in these important tasks,” said study author Fahmi Hassan Fadhel, an associate professor of clinical psychology at Qatar University. “The superiority of artificial intelligence in the areas of perceiving and understanding people’s emotions may mean that it will perhaps be more useful than a human psychotherapist, which is a very concerning issue.”

The study included 180 male psychologists from King Khalid University in Saudi Arabia, divided based on their educational status into bachelor’s and doctoral students. The AI participants included some of the most advanced LLMs available: OpenAI’s ChatGPT-4, Google Bard, and Microsoft Bing.

Each participant, both human and AI, was asked to respond individually to 64 scenarios presented in the Social Intelligence Scale. This scale was chosen because it is well-established and offers a reliable measure of the social skills that are crucial in psychotherapy. The responses were collected and scored according to predefined criteria.

The items were designed to measure two primary dimensions of social intelligence: the soundness of judgment of human behavior and the ability to act wisely in social situations. The soundness of judgment involves understanding social experiences through observation of human behavior, while the ability to act pertains to analyzing social problems and choosing appropriate solutions.

The results indicated a significant variance in the performance of different AI models and human psychologists, suggesting that some AI systems have advanced to a point where they can outperform human professionals in specific aspects of social intelligence.

Among the AI models evaluated, ChatGPT-4 stood out by demonstrating the highest level of social intelligence. It scored 59 out of 64 on the Social Intelligence Scale, effectively surpassing the performance of all human psychologists in the study. The average social intelligence scores were 39.19 for bachelor’s students and 46.73 for doctoral students.

On the other hand, Bing also performed well, scoring 48 out of 64. This score indicated that Bing outperformed 90% of the bachelor’s students and was on par with 50% of the doctoral students.

In contrast, Google Bard exhibited a lower level of social intelligence in this study. It scored 40 out of 64, which positioned it roughly equivalent to the bachelor-level psychologists but significantly below doctoral students.

The findings serve as a benchmark for future development of AI systems intended for psychotherapy and counseling. Knowing that AI can match or even exceed human performance in social intelligence tasks provides a strong foundation for further integrating these technologies into mental health services.

“The study provides a quick overview of the rapid developments in artificial intelligence,” Fadhel told PsyPost. “It’s a bright outlook for the near future.”

However, the study also raises important questions about training, development, and the ethical use of AI in sensitive areas like mental health, where the ability to empathize and form therapeutic relationships is traditionally viewed as uniquely human.

“Perhaps the biggest caveats will relate to the capabilities of artificial intelligence in the future to understand and analyze human feelings and make decisions based on that,” Fadhel said. “We do not know where developments in this field are headed. To date, the controls imposed on artificial intelligence developers are still at their lowest levels, according to our knowledge.”

The study, “Artificial intelligence and social intelligence: preliminary comparison study between AI models and psychologists,” was authored by Nabil Saleh Sufyan, Fahmi H. Fadhel, Saleh Safeer Alkhathami, and Jubran Y. A. Mukhadi.

TweetSendScanShareSendPin1ShareShareShareShareShare

RELATED

Trump’s speeches stump AI: Study reveals ChatGPT’s struggle with metaphors
Artificial Intelligence

Trump’s speeches stump AI: Study reveals ChatGPT’s struggle with metaphors

July 15, 2025

Can an AI understand a political metaphor? Researchers pitted ChatGPT against the speeches of Donald Trump to find out. The model showed moderate success in detection but ultimately struggled with context, highlighting the current limits of automated language analysis.

Read moreDetails
Daughters who feel more attractive report stronger, more protective bonds with their fathers
Artificial Intelligence

People who use AI may pay a social price, according to new psychology research

July 14, 2025

Worried that using AI tools like ChatGPT at work makes you look lazy? New research suggests you might be right. A study finds employees who use AI are often judged more harshly, facing negative perceptions about their competence and effort.

Read moreDetails
Is ChatGPT really more creative than humans? New research provides an intriguing test
ADHD

Scientists use deep learning to uncover hidden motor signs of neurodivergence

July 10, 2025

Diagnosing autism and attention-related conditions often takes months, if not years. But new research shows that analyzing how people move their hands during simple tasks, with the help of artificial intelligence, could offer a faster, objective path to early detection.

Read moreDetails
Positive attitudes toward AI linked to problematic social media use
Artificial Intelligence

Positive attitudes toward AI linked to problematic social media use

July 7, 2025

A new study suggests that people who view artificial intelligence positively may be more likely to overuse social media. The findings highlight a potential link between attitudes toward AI and problematic online behavior, especially among male users.

Read moreDetails
Stress disrupts gut and brain barriers by reducing key microbial metabolites, study finds
Artificial Intelligence

Dark personality traits linked to generative AI use among art students

July 5, 2025

As generative AI tools become staples in art education, a new study uncovers who misuses them most. Research on Chinese art students connects "dark traits" like psychopathy to academic dishonesty, negative thinking, and a heavier reliance on AI technologies.

Read moreDetails
AI can already diagnose depression better than a doctor and tell you which treatment is best
Artificial Intelligence

New research reveals hidden biases in AI’s moral advice

July 5, 2025

Can you trust AI with your toughest moral questions? A new study suggests thinking twice. Researchers found large language models consistently favor inaction and "no" in ethical dilemmas.

Read moreDetails
Scientists reveal ChatGPT’s left-wing bias — and how to “jailbreak” it
Artificial Intelligence

ChatGPT and “cognitive debt”: New study suggests AI might be hurting your brain’s ability to think

July 1, 2025

Researchers at MIT investigated how writing with ChatGPT affects brain activity and recall. Their findings indicate that reliance on AI may lead to reduced mental engagement, prompting concerns about cognitive “offloading” and its implications for education.

Read moreDetails
Readers struggle to understand AI’s role in news writing, study suggests
Artificial Intelligence

Readers struggle to understand AI’s role in news writing, study suggests

June 29, 2025

A new study finds that readers often misunderstand AI’s role in news writing, creating their own explanations based on limited information. Without clear byline disclosures, many assume the worst.

Read moreDetails

SUBSCRIBE

Go Ad-Free! Click here to subscribe to PsyPost and support independent science journalism!

STAY CONNECTED

LATEST

Scientists identify the brain’s built-in brake for binge drinking

Trump’s speeches stump AI: Study reveals ChatGPT’s struggle with metaphors

Childhood maltreatment linked to emotion regulation difficulties and teen mental health problems

Caffeine may help prevent depression-like symptoms by protecting the gut-brain connection

Secret changes to major U.S. health datasets raise alarms

Moral outrage spreads petitions online—but doesn’t always inspire people to sign them

The triglyceride-glucose index: Can it predict depression risk in the elderly?

People with ADHD exhibit altered brain activity before making high-stakes choices

         
       
  • Contact us
  • Privacy policy
  • Terms and Conditions
[Do not sell my information]

Welcome Back!

Login to your account below

Forgotten Password?

Retrieve your password

Please enter your username or email address to reset your password.

Log In

Add New Playlist

Subscribe
  • My Account
  • Cognitive Science Research
  • Mental Health Research
  • Social Psychology Research
  • Drug Research
  • Relationship Research
  • About PsyPost
  • Contact
  • Privacy Policy