Humans create more novelty than ChatGPT when retelling stories

A recent study explored the differences between how humans and ChatGPT retell stories. The researchers found that while ChatGPT provided concise summaries of original stories with minimal changes in further retellings, humans introduced significant variations and novel elements with each retelling. The paper was published in Scientific Reports.

ChatGPT is an advanced large language model developed by OpenAI that can generate human-like text based on the input it receives and utilizing a vast database of human knowledge it was trained on. This makes it very useful for answering questions, writing essays, providing conversational assistance, and many other purposes. ChatGPT is able to produce coherent and contextually relevant textual responses and this can be used to enhance productivity in many fields of human activity. By automating routine tasks and offering instant information, ChatGPT has the potential to free up human time for more complex and creative endeavors.

The introduction of ChatGPT and other artificial intelligence systems started a revolution in the way human economy and society functions. Their ability to handle large volumes of inquiries and provide personal assistance in a way that only humans could provide until recently enabled individuals to seek and obtain help and knowledge they need at any time. On the other hand, scientists are still exploring how including ChatGPT in communication systems that were completely reliant on humans just a couple of years ago changes the nature of those communications.

Study author Fritz Breithaupt and his colleagues wanted to explore how ChatGPT retells stories compared to how humans do it. They focused on two aspects of retelling stories – the stability of language and affect preservation. Affect preservation refers to whether the retold version of the story maintains the same emotional tone of the situation that was present in the original. The stability of language refers to whether words, concepts, and grammatical constructions from the original are preserved in the retold version.

The study involved participants from Amazon Mechanical Turk (MTurk) who were asked to write short stories of approximately 120-160 words, categorized as happy, mildly happy, mildly sad, or sad (without using explicit emotional words like “happy” or “sad”).

Subsequently, 348 other participants were tasked with retelling these stories or their retold versions. Each participant retold three different stories, resulting in a chain of retellings where each story was retold three times by different individuals. This process created 116 original stories, each with three retellings. Finally, 537 participants rated these stories for their emotional content and other characteristics.

ChatGPT 3 was also used to retell the same stories using identical instructions. To ensure fairness, different ChatGPT accounts were employed for each retelling in the chain, preventing the model from accessing previous retellings. The resulting retellings were then rated by 531 individuals recruited via Prolific.

The results showed that both ChatGPT and humans significantly shortened the stories in their retellings. However, ChatGPT’s retellings were substantially shorter right from the first iteration, with only slight decreases in length in subsequent retellings. Humans, on the other hand, progressively reduced the text length with each retelling, displaying greater variability in the retellings’ lengths.

Analysis of the language used in retellings revealed notable differences. ChatGPT maintained more nouns, adjectives, and prepositions, using words typically acquired later in life. In contrast, humans employed more verbs, adverbs, and negations, favoring language acquired at a younger age. This suggests that human retellings focus more on actions and emotions, while ChatGPT emphasizes descriptions and entities.

Both ChatGPT and humans effectively preserved the emotional tone of the original stories in their retellings. This ability to maintain the core emotional impact highlights a significant similarity between human and machine storytelling, despite differences in how they achieve it.

One of the key findings was that humans displayed ongoing creativity in their retellings, introducing new words and concepts with each iteration. This incremental reduction and addition of novel elements contrast sharply with ChatGPT’s approach, which produces a concise summary in the first retelling and makes few changes thereafter. The study found that human retellings become increasingly novel as the process progresses, with approximately 55%-60% new content in each retelling.

These findings suggest that while ChatGPT can serve as a valuable tool in various applications, it cannot fully replicate the richness and variability of human narrative communication.

“The results reveal that spontaneous retelling by humans involves ongoing creativity, anchored by emotions, beyond the default probabilistic wording of large language models such as ChatGPT,” the study authors concluded.

The study sheds light on the differences between how ChatGPT and humans retell stories. However, it should be noted that both humans and ChatGPT retelling depend on the instructions they receive. A tweak of the instructions given to retellers could produce completely different results.

The paper, “Humans create more novelty than ChatGPT when asked to retell a story,” was authored by Fritz Breithaupt, Ege Otenen, Devin R. Wright, John K. Kruschke, Ying Li, and YiyanTan.