Songbirds possess a sophisticated vocal organ that allows them to produce complex songs. Because of their many similarities to human speech, these songs provide a good model for deciphering the mechanisms responsible for learning and performing vocal skills. Now, Timothy Q. Gentner of the University of California, San Diego, and his team present a system capable of reading the brain activity of these animals and generating melodic vocalizations. The journal Current Biology publishes the results of their work.

Researchers placed silicon electrodes on the skulls of adult male zebra finches (Taeniopygia guttata) to monitor brain activity during singing. Specifically, they recorded the brain waves of the premotor nucleus, involved in the production of sequences of between 3 and 10 syllables in length, which are repeated without variation and make up the songs of these birds.

The brain patterns served as the basis for learning an artificial neural network. From simple representations of the vocalizations, i.e. mathematical equations modeling the changes in pressure and tension of the vocal apparatus, the algorithm was able to generate predictions about the sounds and reproduce them, based on the detected brain electrical activity.

The next step will be to train the system to read brain waves and produce melodies in real time, without the aid of intermediate models or simplifications. However, this presents a major challenge: spatiotemporal factors. Both birds and people hear vocalizations as we utter them, which allows us to adjust our song or speech and decide on the next note or word, as well as to correct errors. Therefore, for future vocal prostheses to be useful, they must allow for this feedback. That is, to interpret brain activity and reproduce sounds quickly, so that the subject can express himself fluently.

Marta Pulido Salgado

Reference: «Neurally driven synthesis of learned, complex vocalizations», E. M. Arneodo et al., en Current Biology. Published on June 16, 2021.