A musical robot, created by researchers at the University of Nottingham to help musicians with disabilities access music-making in a way that hasn't previously been possible, has won two prestigious sound awards.
Jess+ was developed as part of the European Research Council funded Digital Score (DigiScore) project by Craig Vear, Professor of Music and Computer Science, for inclusive music making, using AI and robotics. It is the first large-scale collaboration between the university's Department of Music, School of Computer Science, Orchestras Live and Sinfonia Viva.
The two prestigious awards recently allocated to the project were for the category 'Best Sound Innovation' from the Sound of the Year Awards; and 'Music Tech' award, presented from the Global Tech Awards.
The Sound of the Year Awards celebrate sound in all its innovative and creative forms, from all corners of the world, and are judged by a panel of leading lights from the international sound community. The Global Tech Awards recognise and celebrate the very best in technology, with a focus on innovation, creativity and excellence. They aim to identify and reward the most exceptional technology solutions and services from around the world.
Being involved with the Jess+ project was an honour and a privilege. I got to work with so many amazing colleagues and was supported by wonderful external organisations such as Orchestra's Live and Sinfonia Viva."
The robot was used in a series of workshops involving musicians Jess Fisher (who has a physical disability) and Clare Bhabra and Deirdre Bencsik from Sinfonia Viva. During the workshops, the robot moved and drew marks on paper in response to the real-time sound created by a live musician. The musicians in turn would interpret these movements and marks as notational material and make a sound in response, much like a graphic score.
Professor Vear added: "The three musicians at the centre of this project (Jess, Clare and Deidre) were the true stars as they brought a degree of professionalism and creativity that was beyond my expectations. Their contribution is what has led to these wonderful awards. Being acknowledged by external awards such as these means much more as it clearly indicates that my research has impact beyond academia, which is where I truly value it."
By creating Jess+ the team wanted to build an embodied-AI system that could facilitate co-creation for an improvising ensemble of disabled and non-disabled musicians, so that the expressive qualities of all musicians could be amplified and bound together in a co-creative system.
There are significant barriers to music-making for disabled musicians. One such barrier, is that while non-disabled people can make music in many ways, this is less true for disabled musicians, who need new accessible instruments, new creative processes, and new hierarchies of "success" (Drake Music). Another is how a mixed ensemble of disabled and non-disabled musicians might improvise together in such a way that dexterity is not a limiting factor. A further barrier is how disabled musicians can be excluded from the creative process in which a music score is created for them. These barriers can leave creative potentials untapped, setup hierarchical measurements of creative involvement, and limit the potential input of disabled musicians.
In 2023, a performance from the Jess+ team was held at the BBC studios in London for a performance as part of the Bridging Responsible AI Divides (BRAID) launch event. BRAID is a three-year national research programme which aims to bring together UK-based researchers from across the arts and humanities to drive responsible AI innovation, in partnership with the BBC and the Ada Lovelace Institute.
The video of the performance at the BBC can be viewed here.
