![[Adobe Stock]](https://sp-ao.shortpixel.ai/client/to_webp,q_glossy,ret_img,w_750,h_375/https://www.psypost.org/wp-content/uploads/2023/12/chatgpt-750x375.jpg)
A recent study compared personality perceptions of public figures generated by ChatGPT with those generated by human raters, finding a strong agreement between the two. However, when the researchers controlled for the likeability and demographics of the public figures, the level of agreement decreased somewhat. This research was published in Scientific Reports.
People constantly evaluate the characteristics of others, forming first impressions about strangers almost instantaneously. Studies show that it takes less than a second for a person to decide if someone represents a threat. Judging other characteristics may take a bit longer, but initial perceptions are formed quickly and significantly influence future interactions. These early judgments can impact a person’s success in social situations and life in general, as they shape how their personality is perceived by others.
Perception by others is particularly crucial for public figures. Politicians’ success in elections and their approval ratings largely depend on voter perceptions. Similarly, a company’s reputation, valuation, and overall performance can be influenced by how the public views its CEO or founder. These factors explain why researchers are interested in studying the personality perceptions of public figures.
Study authors Xubo Cao and Michal Kosinski sought to explore how closely GPT-3’s personality perceptions of public figures align with those of human raters. GPT-3, or Generative Pre-trained Transformer 3, is a large language model developed by OpenAI that generates human-like text based on input. It was previously used by the free version of the ChatGPT textbot, which now uses GPT-3.5 and GPT-4.
The researchers selected 300 widely recognized public figures from 43 countries, sourced from the Pantheon 1.0 dataset. Since a disproportionate number of these figures were artists, the researchers limited the number of artists to 100. The remaining public figures were categorized into seven domains: business and law, exploration, humanities, institutions, science and technology, sports, and others.
The names of these public figures were presented to 600 raters recruited via Prolific. Each rater assessed the likeability and Big Five personality traits (using the Ten-Item Personality Inventory – TIPI) of 10 randomly chosen public figures. Raters could skip figures they were unfamiliar with, leading the researchers to remove 74 public figures recognized by fewer than 10 raters.
GPT-3 uses a specific format to store data about the meanings of words, including names of public figures and attributes. The researchers extracted data about the meanings of public figures, known as embeddings, and used them to statistically predict the human raters’ responses.
The results showed that GPT-3’s embeddings accurately predicted human perceptions of the public figures. There was a strong correlation between the actual human responses and the predictions based on GPT-3 embeddings, with more accurate predictions for more popular public figures.
An analysis of the perceptions revealed that female public figures were generally perceived as more agreeable. Additionally, the least likeable figures (e.g., Charles Manson, Lee Harvey Oswald, Kim Jong Il) were often assigned socially undesirable personality traits. To address potential biases, the researchers re-ran their calculations controlling for likeability and demographics, finding that the predictions were slightly less precise but still accurate.
“Our results indicate that public figures’ perceived personality can be accurately predicted from their names’ location in GPT-3’s semantic space. Our models remained accurate even when controlling for public figures’ demographics and overall likability. Moreover, the models showed high face validity as revealed by the examination of public figures predicted to score at the top/bottom of each of the traits, as well as the personality-descriptive adjectives occupying the models’ extremes,” the study authors concluded.
This study tests GPT-3’s accuracy in predicting human perceptions of public figures’ personalities. However, it is important to note that public perceptions can change over time, while GPT-3’s data does not update after training. Companies that run large language models like GPT-3 periodically release new versions trained on updated data. Finally, public perceptions of a figure’s personality may not always reflect their true personality traits.
The study, “Large language models know how the personality of public figures is perceived by the general public,” was authored by Xubo Cao and Michal Kosinski.