In May 2008, Honda sent a robot to conduct the Detroit Symphony Orchestra. Although predominantly a publicity exercise to promote Honda’s sponsorship of the Orchestra, ASIMO’s turn as robotic maestro potentially highlights some of the challenges facing researchers in high-performance human-computer interaction (HCI) environments. This paper proposes that conductors (whether human or robot) need to be able to not only perform the non-verbal behaviour through which musicians communicate but also need to be able to receive and understand the non-verbal ‘feedback’ that musicians give conductors and each other via gesture, tool-use and music performance. ASIMO (as an example) needs to be able to dynamically (and non-verbally) respond to simultaneous and, potentially, conflicting or contrasting musician non-verbal behaviour.
This paper proposes that the first step in this direction needs to be a comprehensive understanding of both conductor and musician behaviour. Rather than examining novel human interaction with technology designed to create or manipulate sonic environments in experimental settings (such as in Williams, Kabisch & Dourish 2005), this paper seeks to analyse the way that musicians, in an ensemble, interact with each other in real-world situations, as a basis for improving HCI in music settings. This analysis will identify the multiple modes across which musicians act and interact, and how non-verbal behaviours outside of a music environment inform the gestures used and understood across the orchestra. This linguistic-based perspective will have critical implications both for ASIMO, and other forms of technology needed by musicians to continue to innovate.
Authors: Katharine Parton
Event: SF08: Embodied Interaction in Mobile, Physical and Virtual Environments Workshop