Robotic musicianship

30 November 2011

share this:

Shimon is an interactive robotic marimba player that can improvise both music and choreography in real-time to the melody of a human pianist.

Playing an instrument does not make you a musician. To become a musician you need to listen, analyze, improvise, and interact through the sound you produce and your body language.

With this in mind, Hoffman et al. explore robotic musicianship. Unlike robots that simply perform a sequence of notes, Shimon’s performances are composed of a sequence of gestures that may or may not produce sound. Using gestures as the building blocks of musical expression is particularly appropriate for robotic musicianship, and nicely fits with our embodied view of human-robot interaction.

The robot is able to improvise by following basic aspects of standard Jazz joint improvisation and can anticipate gestures to easily synchronize with duet partners. Building on this, the human and robot could perform three types of interactions. In the first interaction, the robot and human played two distinct musical phrases, where the second phrase is a commentary on the first phrase. The second interaction was centered around the choreographic aspect of movement with the notes appearing as a “side-effect” of the performance. The third interaction was a rhythmic phrase-matching improvisation.

Using this improvisation system, the pair performed full-length performances of nearly 7 minutes in front of live public audiences and more than 70’000 online viewers.

After the live performances, additional experiments were conducted to investigate the importance of physical embodiment and visual contact in Robotic Musicianship. Results show that synchronization between the robot and musician can be aided by visual contact when the tempo is uncertain and slow. In addition, the audience perceives Shimon as playing better, more like a human, as more responsive, and even more inspired when compared to a “computer musician”. Shimon was also rated as better synchronized, more coherent, communicating, and coordinated; and the human as more inspired and more responsive.

In the future, Hoffman et al. hope to further explore robot musicianship by giving Shimon a socially expressive robot head, vision and new gestures.

tags: ,

Sabine Hauert is President of Robohub and Associate Professor at the Bristol Robotics Laboratory
Sabine Hauert is President of Robohub and Associate Professor at the Bristol Robotics Laboratory

Related posts :

The one-wheel Cubli

Researchers Matthias Hofer, Michael Muehlebach and Raffaello D’Andrea have developed the one-wheel Cubli, a three-dimensional pendulum system that can balance on its pivot using a single reaction wheel. How is it possible to stabilize the two tilt angles of the system with only a single reaction wheel?
30 June 2022, by and

At the forefront of building with biology

Raman is, as she puts it, “a mechanical engineer through and through.” Today, Ritu Raman leads the Raman Lab and is an Assistant Professor in the Department of Mechanical Engineering.
28 June 2022, by

Hot Robotics Symposium celebrates UK success

An internationally leading robotics initiative that enables academia and industry to find innovative solutions to real world challenges, celebrated its success with a Hot Robotics Symposium hosted across three UK regions last week.
25 June 2022, by

Researchers release open-source photorealistic simulator for autonomous driving

MIT scientists unveil the first open-source simulation engine capable of constructing realistic environments for deployable training and testing of autonomous vehicles.
22 June 2022, by

In this episode, Audrow Nash speaks to Maria Telleria, who is a co-founder and the CTO of Canvas. Canvas makes a drywall finishing robot and is based in the Bay Area. In this interview, Maria talks ab...
21 June 2022, by and

Coffee with a Researcher (#ICRA2022)

As part of her role as one of the IEEE ICRA 2022 Science Communication Awardees, Avie Ravendran sat down virtually with a few researchers from academia and industry attending the conference.

©2021 - ROBOTS Association


©2021 - ROBOTS Association