Robohub.org
 

Robotic musicianship


by
30 November 2011



share this:

Shimon is an interactive robotic marimba player that can improvise both music and choreography in real-time to the melody of a human pianist.

Playing an instrument does not make you a musician. To become a musician you need to listen, analyze, improvise, and interact through the sound you produce and your body language.

With this in mind, Hoffman et al. explore robotic musicianship. Unlike robots that simply perform a sequence of notes, Shimon’s performances are composed of a sequence of gestures that may or may not produce sound. Using gestures as the building blocks of musical expression is particularly appropriate for robotic musicianship, and nicely fits with our embodied view of human-robot interaction.

The robot is able to improvise by following basic aspects of standard Jazz joint improvisation and can anticipate gestures to easily synchronize with duet partners. Building on this, the human and robot could perform three types of interactions. In the first interaction, the robot and human played two distinct musical phrases, where the second phrase is a commentary on the first phrase. The second interaction was centered around the choreographic aspect of movement with the notes appearing as a “side-effect” of the performance. The third interaction was a rhythmic phrase-matching improvisation.

Using this improvisation system, the pair performed full-length performances of nearly 7 minutes in front of live public audiences and more than 70’000 online viewers.

After the live performances, additional experiments were conducted to investigate the importance of physical embodiment and visual contact in Robotic Musicianship. Results show that synchronization between the robot and musician can be aided by visual contact when the tempo is uncertain and slow. In addition, the audience perceives Shimon as playing better, more like a human, as more responsive, and even more inspired when compared to a “computer musician”. Shimon was also rated as better synchronized, more coherent, communicating, and coordinated; and the human as more inspired and more responsive.

In the future, Hoffman et al. hope to further explore robot musicianship by giving Shimon a socially expressive robot head, vision and new gestures.



tags: ,


Sabine Hauert is President of Robohub and Associate Professor at the Bristol Robotics Laboratory
Sabine Hauert is President of Robohub and Associate Professor at the Bristol Robotics Laboratory





Related posts :



Robots to the rescue: miniature robots offer new hope for search and rescue operations

  09 Sep 2025
Small two-wheeled robots, equipped with high-tech sensors, will help to find survivors faster in the aftermath of disasters.

#IJCAI2025 distinguished paper: Combining MORL with restraining bolts to learn normative behaviour

and   04 Sep 2025
The authors introduce a framework for guiding reinforcement learning agents to comply with social, legal, and ethical norms.

Researchers are teaching robots to walk on Mars from the sand of New Mexico

  02 Sep 2025
Researchers are closer to equipping a dog-like robot to conduct science on the surface of Mars

Engineering fantasy into reality

  26 Aug 2025
PhD student Erik Ballesteros is building “Doc Ock” arms for future astronauts.

RoboCup@Work League: Interview with Christoph Steup

and   22 Aug 2025
Find out more about the RoboCup League focussed on industrial production systems.

Interview with Haimin Hu: Game-theoretic integration of safety, interaction and learning for human-centered autonomy

and   21 Aug 2025
Hear from Haimin in the latest in our series featuring the 2025 AAAI / ACM SIGAI Doctoral Consortium participants.

AIhub coffee corner: Agentic AI

  15 Aug 2025
The AIhub coffee corner captures the musings of AI experts over a short conversation.

Interview with Kate Candon: Leveraging explicit and implicit feedback in human-robot interactions

and   25 Jul 2025
Hear from PhD student Kate about her work on human-robot interactions.



 

Robohub is supported by:




Would you like to learn how to tell impactful stories about your robot or AI system?


scicomm
training the next generation of science communicators in robotics & AI


 












©2025.05 - Association for the Understanding of Artificial Intelligence