A new program developed by聽Rochester researchers uses machine learning to allow users to improvise duets with an artificial intelligence partner.
To play a duet with German composer Johann Sebastian Bach, you don鈥檛 have to travel back to the 18th century; thanks to a new program developed by researchers at the 人妻少妇专区, you only need a computer.
The web-based program, called , was developed by , an associate professor of electrical and computer engineering and of computer science, and members of his lab, including Yongyi Zang 鈥�23 and PhD student Christodoulos Benetatos. BachDuet allows a person to improvise duets in the style of Bach with an artificial intelligence (AI) counterpoint in real time. By visiting the BachDuet website, a user can play duets with the AI agent using a computer keyboard, mouse, touchscreen, or MIDI keyboard.
In BachDuet, the AI agent will either follow the human musician, lead the improvisation, or 鈥渏ust goof around,鈥� Duan says.
But the researchers foresee the program could have applications in music education, entertainment, and music therapy. For instance, says Duan, 鈥渕ost of the existing musical therapies are listening-based, but in recent years we have seen some playing-based music therapies that have been shown to help improve cognitive functions.鈥�
BachDuet is part of Duan鈥檚 research project, funded by a National Science Foundation Faculty Early Career Development (CAREER) award, to develop algorithms and systems that allow humans to work with machines to harmonize pitch, coordinate musical timing, and convey expression in music.
His lab uses signal processing, artificial intelligence, and machine learning algorithms, which can empower machines with music perception skills, more expressive performance, and deeper 鈥渦nderstanding鈥� of music theory and composition. For this project, the team input hundreds of Bach鈥檚 compositions and used machine learning techniques to allow the program to 鈥渟tudy鈥� the music. During improvisation, the program assumes a 4/4 time signature and makes a decision about which note to trigger or sustain.
鈥淭he goal of our project is to develop tools that allow humans to play music with computer systems in a way similar to human musicians collaborating with each other,鈥� Duan says. 鈥淲ith BachDuet, we wanted to develop a tool that allows musicians and novices alike to practice counterpoint improvisation.鈥�
鈥淎nd,鈥� he adds, 鈥渢o have fun.鈥�
Read more
CAREER awards spur junior researchers along varied paths
Four Rochester researchers from four different fields are 2019 recipients of the National Science Foundation鈥檚 most prestigious recognition for junior faculty members.
Giving virtual reality a 鈥榲isceral鈥� sound
Using recital halls as their 鈥渓abs,鈥� and recording some of the best music students in the world, University researchers are creating immersive virtual reality concert videos.
The mysteries of music鈥攁nd the key of data
Researchers at the University are at the cutting edge of the intersection of data science and music: developing databases to study music history and creating algorithms to automatically identify a genre or singer.
