A recent choral concert reflecting on the past with the theme 鈥淢emories,鈥� also featured the debut of a new technological advancement in music.
The 听presented a concert that included Chinese and English art and folk songs and also marked the launch of an automatic lyrics display system developed by Zhiyao Duan, an assistant professor of at the 人妻少妇专区, and his PhD student, Bochen Li. The system is able to follow a performance and display lyrics on a large screen in real time.
During the May concert, the system displayed the lyrics in both Chinese and English to a multicultural audience composed of people with a variety of native languages.
鈥淢usical performance is all about communication between musicians and the audience,鈥� Duan says. 鈥淭his system helps musicians convey ideas鈥攊n this case the lyrics鈥攖o the audience and helps the audience better appreciate the performance.鈥�
Duan sings tenor with the Chinese Choral Society and brainstormed with Youti Kuo, secretary of the choral society, and Ming-Lun Lee, an assistant professor of audio and music engineering at the 人妻少妇专区 and the director of the choral society, about ways he could also contribute his knowledge as a researcher in artificial intelligence and music.
The choir performs in various languages, and previously, in order to display lyrics in real time, a person very familiar with the songs would have to manually type the words during the live performance. Duan鈥檚 innovation allows an audience to view the lyrics at the exact moment they are performed, without the need for manual lyric input during the performance.
鈥淚t is very important to have the audience understand the lyrics in foreign languages,鈥� says Lee. 鈥淲ith the help of the automatic lyrics displaying system, we can train the system before the concert and avoid human errors.鈥�
Duan鈥檚 system uses a dynamic time warping algorithm, which reconciles differences between two sequences that may vary in speed. For the system to work, it needs a reference recording to compare with the live performance. The choral society taped a reference recording during a rehearsal and pre-aligned the lyrics to this recording. During the live performance, the algorithm aligned the live performance with the reference recording and displayed the lyrics in real time.
Although manual work is still required to add lyrics to the reference recording prior to the concert, it can be completed in a much more flexible fashion: it can be done any time before the concert and allows for revisions. In addition, once the lyrics alignment is completed, it can be used as a reference for any live performance of the same piece.
鈥淲e hope this will soon be available to choirs worldwide, either for commercial use or for free,鈥� Duan says. 鈥淲e are working on building a system and a repertoire that are scalable.鈥�
In addition to the debut of the lyrics system, the concert featured the US premiere of the suite听A Journey Named Life听for chorus, flute, and piano, composed by Ching-Mei Lin 鈥�06 (MM).
