In a study led by Jii Kwon and Chun Kee Chung, from Seoul National University, researchers explored whether they could recreate melodies from the brain activity of people imagining how songs continue after listening to a few bars. The researchers' work is published in eNeuro.
Study participants were 10 epilepsy patients with electrodes on the surfaces of their brains for clinical monitoring purposes. Participants quietly listened to the first bars of children's tunes, then imagined the melodies of subsequent bars of the songs before humming their imagined melodies. Brain activity from the patients carried information about relative pitches within the imagined melodies. Combining note-level pitch predictions over time allowed the research team to recreate melodic changes that reflected the structure of the imagined songs. Kwon elaborates, "The model we created decodes relative pitch classes—such as do, re, mi, fa, sol, and la—rather than exact, absolute pitches. This is useful because people often recognize melodies by the relationships between notes, even when the same melody is played in a different key."
According to the authors, this study provides evidence that brain signals reveal the pitches of notes in imagined musical melodies. These findings may motivate future work on decoding other musical attributes, such as tempo and dynamics, or exploring whether similar approaches can extend beyond familiar children's songs to more complex music. Kwon hopes these findings may eventually help develop ways of reconstructing imagined melodies for individuals with little music background or even patient populations—such as those with Amyotrophic Lateral Sclerosis—who cannot articulate what they are imagining.
