Artificial intelligence helps decode the neuroscience of dance

Dance engages the human mind in a way few other activities can, merging the rhythmic perception of sound with the visual appreciation of movement. A new study uses advanced artificial intelligence to decode how the brain processes this complex art form, revealing that computer models can accurately predict neural activity in human observers. The research suggests that expert dancers process these performances differently than novices, displaying a surprisingly high level of diversity in their neural responses.

Scientists have attempted to understand how the brain interprets sensory information for decades. Traditional experiments often rely on simple, isolated stimuli to maintain control over the data. A researcher might expose a participant to a single tone or a flashing light to see which neurons fire. While this approach offers precision, it often fails to capture the complexity of real-world experiences where senses blend continuously. The human brain rarely processes sound without context or sight without accompanying noise.

To address this limitation, a team of researchers turned their attention to dance. This medium naturally fuses dynamic body movement with music, requiring the observer to integrate visual and auditory signals simultaneously. Previous neuroscience studies on dance often separated these elements. They might show participants silent videos of movement or play music without visual accompaniment. Such methods sever the intricate connection between a beat and a step, which is the defining characteristic of dance.

The research team, led by Yu Takagi and Hiroshi Imamizu at the University of Tokyo, sought to bridge this gap using modern technology. They aimed to determine if advanced computer programs could mimic the way the human brain perceives these combined sensations. They also wanted to see if the brain activity of a professional dancer differed significantly from that of someone with no dance training.

The investigators recruited fourteen participants for the experiment. Half of the group consisted of expert dancers with more than five years of training in various genres. The other half included novices who had no formal dance background. These individuals watched five hours of video clips featuring various styles of street dance, such as hip-hop, lock, and pop. They viewed these clips while inside a functional magnetic resonance imaging scanner. This machine tracks blood flow in the brain to measure activity in real time.

Free daily newsletter

To analyze the massive amount of data collected, the team utilized a deep generative artificial intelligence model called EDGE. This computer program is designed to create realistic dance choreography. It works by analyzing a music track and predicting the physical movements that should accompany it. The model effectively hallucinates a dance routine based on the audio input it receives.

The researchers extracted mathematical features from the artificial intelligence model. These features represented motion, audio, and the combined cross-modal information. The team then built encoding models to see which of these features best predicted the actual brain activity recorded in the participants. They found that the cross-modal features explained the brain activity more accurately than the motion or audio features alone.

This predictive success was most evident in the high-level association areas of the brain. These are the regions responsible for combining different types of sensory information. The results indicate that the brain does not just see a moving body and hear a song as separate events. Instead, it processes the interaction between the two as a distinct, unified phenomenon. The artificial intelligence model operates by predicting the next movement in a sequence, and its success in predicting brain activity suggests that the human brain may function similarly.

The study also highlighted distinct differences between the two groups of participants. The brain activity of expert dancers was more accurately predicted by the dance features than that of the novices. This implies that the experts possess a neural framework that is more finely tuned to the nuances of choreography. However, the experts also exhibited greater variability among themselves.

While the brain patterns of the novices looked relatively similar to one another, each expert processed the performance in a unique way. This finding challenges the assumption that expertise leads to a uniform way of seeing the world. Instead, deep knowledge appears to allow for a more personalized interpretation of the art.

Professor Hiroshi Imamizu commented on this unexpected result. “Surprisingly, compared to nonexpert audiences, our brain-activity simulator was able to more precisely predict responses in experts. Even more interesting was the fact that while nonexperts exhibited individual differences in response patterns, the videos elicited a more diverse number of patterns in experts.”

Beyond the mechanics of movement, the team explored how the brain encodes the emotional content of dance. They asked a large separate group of people to rate the video clips on qualities such as aesthetics, dynamics, and boredom. The researchers then mapped these subjective ratings onto the brain data using their computational model. They found that specific emotional concepts correlated with activity in distinct brain networks.

Feelings of boredom were associated with reduced activity in the default mode network. This is a set of brain regions that is typically active when the mind is wandering or at rest. Conversely, the perception of dynamic movement triggered increased activity in these same areas. Aesthetic appreciation was linked to activity in both the visual cortex and higher-level processing areas. This suggests that enjoying a dance involves a conversation between basic visual perception and complex evaluative thought.

The team tested their model further by creating artificial dance clips. They took the original dance motions and paired them with music from different genres. They fed these mismatched clips into their simulation to estimate how the brain would respond. The simulation predicted that matching music and motion activates sensory regions of the brain more strongly. Mismatched pairings appeared to engage frontal areas of the brain, possibly reflecting the detection of an error or incongruence.

There are limitations to the study that frame how these results should be interpreted. The stimuli focused primarily on street dance, a genre where movement and music are tightly coupled. It is not yet clear if these findings would apply to contemporary dance or other forms with looser connections between sound and choreography. Additionally, the participants watched the videos while lying motionless in a scanner. This passive observation differs from the experience of watching a live performance in a theater.

The researchers also note that the participants were observing dance rather than performing it. Understanding the neural activity of dancers while they are actually moving remains a goal for future science, though it presents significant technical challenges. Despite these hurdles, the study represents a significant step forward in linking computational models with human artistic experience.

The team hopes to use these findings to bridge the gap between neuroscience and the arts. They envision a future where these tools could assist choreographers or help explain the universal appeal of dance. “We would love nothing more than to see our developed brain-activity simulator be used as a tool to create new dance styles which move people,” Imamizu said.

The study, “Cross-modal deep generative models reveal the cortical representation of dancing,” was authored by Yu Takagi, Daichi Shimizu, Mina Wakabayashi, Ryu Ohata, and Hiroshi Imamizu.