A study coordinated by the UAB analyses the physiological and emotional response of 88 persons when watching the same video but accompanied by different types of music: human compositions and AI-generated music. The results, published in the journal PLOS One, reveal that AI can generate music that is perceived to be more exciting, which can have significant implications for the future of audiovisual production.
Generative artificial intelligence (Gen AI) is transforming the world of creativity, and music is no exception. A study coordinated by the Neuro-Com Research Group from the UAB Department of Audiovisual Communication and Advertising, with the involvement of the RTVE Institute in Barcelona and the University of Ljubljana, Slovenia, and recently published in PLOS One, explores a key question: Can AI-generated music produce the same emotional responses as human-composed music in audiovisual contexts?
The research compared the reactions of people when watching videos accompanied by different types of music. During the experiment, 88 participants watched audiovisual clips with identical visuals, but three different sound conditions: music composed by humans, AI-generated music with complex and detailed prompt, and AI-generated music with a simple and less detailed prompt. While participants watched the clips, their physiological responses (such as pupil dilation, blinking and galvanic skin response) and their self-reported emotional reactions were measured.
The results of the experiment were revealing. The AI-generated music triggered greater pupil dilation, indicating a higher level of emotional arousal. The AI music created with sophisticated prompts caused more blinking and changes in the skin, associated with a higher cognitive load. Therefore, differences were observed in the impact caused by the music according to the complexity of the prompt used. Finally, at the emotional level, participants described AI-generated music as more exciting, although human music was perceived as more familiar.
These insights could have significant implications for the future of audiovisual production, particularly regarding the customisation of music to match visual narratives, resource efficiency in creative processes, and the possibility of fine-tuning the emotional impact of content with greater precision using automated tools.
Nikolaj Fišer, first author of the study, points out that "both types of AI-generated music led to greater pupil dilation and were perceived as more emotionally stimulating compared to human-created music". This physiological response is commonly associated with higher levels of emotional arousal. The researcher also states that "our findings suggest that decoding the emotional information in AI-generated music may require greater cognitive effort." This implies that, although AI is capable of producing emotionally powerful music, the way such music is processed by the human brain may differ in terms of mental load and emotional interpretation mechanisms —posing new challenges for designing effective sensory experiences in audiovisual media. This type of research opens new avenues for the study of the perception of sound stimuli generated by advanced algorithmic systems.
This study was conducted by Nikolaj Fišer, PhD student at the Universitat Autònoma de Barcelona (UAB) and the University of Ljubljana; Miguel Ángel Martín-Pascual from the RTVE Institute in Barcelona, Spain; and Celia Andreu-Sánchez, director of the Neuro-Com Research Group at the UAB.
Original article:
