In a world increasingly captivated by the possibilities of artificial intelligence (AI), a recent study by researchers at the University of California, Berkeley, presents a nuanced view of AI’s capabilities and limitations. Their findings, detailed in Perspectives on Psychological Science, suggest that despite AI’s impressive ability to generate text and images, it falls short in a crucial area that defines human intelligence: the capacity for innovation.
The study was motivated by the rapid advancements in AI, particularly in large language models like OpenAI’s ChatGPT. These systems have demonstrated remarkable abilities, from creating engaging narratives to producing intricate visual art.
However, the researchers argue that viewing these AI systems as sentient agents with individual intelligence might be misleading. Instead, they propose considering AI as a powerful new form of cultural technology, akin to writing or the internet, which significantly enhances access to and transmission of knowledge.
The study revolved around two primary components: an “imitation” component and an “innovation” component, each tailored to test the subjects’ abilities to recognize conventional uses of objects and to innovate new uses for them, respectively.
In the “imitation” part of the study, participants were presented with objects and asked to select which ones would best complement a given tool, relying on their understanding of conventional associations between objects. This component was designed to assess the ability to recognize and replicate existing knowledge and associations, a task at which AI systems, trained on vast datasets, are expected to excel due to their proficiency in identifying patterns and correlations.
The “innovation” component, however, posed a more significant challenge: participants were given a problem-solving scenario where they had to achieve a goal without the typical tool at their disposal. Instead, they were provided with alternative objects, among which was one that, despite its superficial dissimilarity, could achieve the goal due to its functional properties. This test was critical in assessing the ability to transcend conventional associations and apply creative thinking to utilize objects in novel ways to solve problems.
The study’s participants included both children, aged 3 to 7 years, and adults, thus encompassing a wide range of human cognitive abilities. These human participants were contrasted with several state-of-the-art AI models, including OpenAI’s GPT-4, among others. To ensure a fair comparison, the AI models were presented with textual descriptions of the scenarios, mimicking the input given to human participants.
The findings revealed a notable divergence in the capabilities of humans and AI. “Even young human children can produce intelligent responses to certain questions that [language learning models] cannot,” explained study author Eunice Yiu. “Instead of viewing these AI systems as intelligent agents like ourselves, we can think of them as a new form of library or search engine. They effectively summarize and communicate the existing culture and knowledge base to us.”
Both children and adults demonstrated a significant capacity for innovation, often choosing the functionally relevant but superficially dissimilar object to solve the problem at hand. This indicated not only an ability to recognize conventional associations but also to innovate beyond them, applying abstract thinking to perceive and utilize the latent functional properties of objects.
“Children can imagine completely novel uses for objects that they have not witnessed or heard of before, such as using the bottom of a teapot to draw a circle,” Yiu said. “Large models have a much harder time generating such responses.”
Conversely, AI models, while adept at identifying superficial associations between objects (the imitation component), showed a marked deficiency in the innovation component. When tasked with selecting objects for novel uses, AI systems frequently defaulted to conventional associations, lacking the human-like capacity to infer new functional applications for these objects.
This was particularly evident in their inability to choose the unconventional yet functionally apt object for the task, highlighting a fundamental gap in AI’s capacity for innovative problem-solving.
These results highlight the limitations of current AI systems in mimicking the full spectrum of human cognitive abilities, particularly when it comes to innovation. While AI can replicate known patterns and associations with remarkable efficiency, its ability to forge new paths and conceive of uncharted applications for existing knowledge remains a challenge.
“AI can help transmit information that is already known, but it is not an innovator,” Yiu said. “These models can summarize conventional wisdom but they cannot expand, create, change, abandon, evaluate, and improve on conventional wisdom in the way a young human can.”
The study, “Transmission Versus Truth, Imitation Versus Innovation: What Children Can Do That Large Language and Language-and-Vision Models Cannot (Yet),” was authored by Eunice Yiu, Eliza Kosoy, and Alison Gopnik.