A program developed at Rensselaer Polytechnic Institute (RPI) allows students with severe physical disabilities to create music just by moving their heads. A digital video camera tracks the students’ movements on a computer screen and translates them into piano scales or drum beats. The program’s developers hope it will open a whole new world of creativity for physically challenged individuals.

Using subtle motions of her head and this newly developed computer program, 16-year-old Annemarie grinned with the realization that she was making music. The teenager, a student at the Rehab School in Poughkeepsie, N.Y., was demonstrating the program for a newspaper reporter recently. Severely physically disabled, Annemarie can’t walk or speak and has little control over the movements of her head and arms.

Annemarie is one of three students at the facility who are being given a rare opportunity to create something of their own, using new computer technology and music therapy.

Music therapy, widely considered a valuable tool for emotional and physical therapy, has not always been accessible for everyone. With the “Adaptive Use Musical Instruments for the Physically Challenged” project at the Rehab School, however, its founders are hoping to bring musical instruments to those whose disabilities prevent them from playing regular instruments. They’re hoping the project not only will give students a chance for creative expression but also will provide a wealth of other valuable experiences.

Musician and RPI professor Pauline Oliveros and drummer Leaf Miller had for years discussed the idea of bringing music to those too severely disabled to play any standard instruments.

Miller, an occupational therapist at the Rehab School, a facility for the physically disabled, had long looked for a way to bring her love of music to the kids she worked with. She’d started a drumming class with the kids about a year-and-a-half ago, but she wanted to find a way for those unable to beat a drum, such as those with cerebral palsy, to join in the experience.

“Playing music isn’t something that’s typically accessible for severely disabled children,” said Miller. “Opening up this opportunity for them is amazing. This really is helping them to gain a certain amount of control over their bodies, which is just great.”

Oliveros is also the founder of Kingston, N.Y.-based musicians organization, the Deep Listening Institute. Through her connections, she was able to secure a $20,000 grant in February for her proposed project. One of her students, Zane Van Dusen, began working in December to develop a computer program that could help the students “play” music using the little range of movement they have.

Van Dusen, who was a double major in computer science and electronic media arts and communication, came up with the idea of using a digital video camera to display the child’s image on the computer screen. A cursor is “placed” on a portion of the student’s head, such as the tip of the nose, and then follows the student’s movements. As it does, it produces music notes—either in piano mode or percussive mode. Moving the head completely in one direction produces a scale in piano mode, while percussive mode creates a serious of quick drum beats or a drum roll.

The system was first tested with the students in May, and remote robotic instruments are now being tested as well. RPI faculty members and former students have been working to construct and program the devices and implement controllers.

Oliveros asked Miller to pick three of the most physically limited students to try out their research. Annemarie, 11-year-old Billy, and Geoffrey, also 11, use wheelchairs, are unable to speak, and have little or no control over the movement of their arms or hands.

Geoffrey, who has been working with the staff at Rehab since he was a little over 2 years old, is also visually impaired. For his mother, Tarez Eisen, seeing her son learning how to “play” an instrument was something she’d never expected.

“The first time I saw him do this, it just blew me away,” said Eisen. “Anything independent that you can do, especially music, is just wonderful.”

Several companies in the U.S., England, and Canada already sell adaptive musical aids, such as special instrument holders and modified drum sticks. But for some of the most severely disabled students, most of these aids still required more controlled movement than they could manage. Like the Deep Listening Institute, some other organizations, including London-based Drake Music Project and Bronx-based Institute for Music and Neurological Function, are also working with new software programs to enable physically challenged students to play music.

At the Institute for Music and Neurological Function, a lot of work is being done with Musical Interface Digital Instrument equipment. MIDI, a processing system, is incorporated into electronic instruments for use with computer programs, according to Executive Director Connie Tomaino.

Unlike with some music therapy, the goal at the Rehab School is eventually to begin “composing,” providing a rare mode of expression for these students, said Oliveros, who said she believes teaching the creative process is the most important aspect of the project.

In music therapy, the goals are usually nonmusical, said Al Bumanis, a spokesman for the American Music Therapy Association.

“The client or patient doesn’t have to be a musician to participate. The goal is not usually a performance, it’s increasing communication skills, understanding, relearning lost skills. Usually the goals are specific to the client’s individual treatment plan,” he said, adding that sometimes a performance is a byproduct and you discover that a patient does take musicianship to a different level.

“From my point of view, making something empowers. That can be very healing, and exciting,” said Oliveros. “In [a lot of] music therapy, there’s no empowerment for patients.”

Eisen agreed: “We have a tendency to focus on the physical therapy, but we sort of forget about the creative stuff,” she said.


Rensselaer Polytechnic Institute

“Adaptive Use Musical Instruments for the Physically Challenged” project

Video demonstration of the adaptive-use music project