Movement to emotions to music: using whole body emotional expression as an interaction for electronic music generation

Alexis Clay (1) , Nadine Couture (1) , Elodie Decarsin (1) , Myriam Desainte-Catherine (2) , Pierre-Henri Vulliard (2) , Joseph Larralde (2)

1 ESTIA INSTITUTE OF TECHNOLOGY
2 LaBRI - Laboratoire Bordelais de Recherche en Informatique

The augmented ballet project aims at gathering research from several fields and directing them towards a same application case: adding virtual elements (visual and acoustic) to a dance live performance, and allowing the dancer to interact with them. In this paper, we describe a novel interaction that we used in the frame of this project: using the dancer's movements to recognize the emotions he expresses, and use these emotions to generate musical audio flows evolving in real-time. The originality of this interaction is threefold. First, it covers the whole interaction cycle from the input (the dancer's movements) to the output (the generated music). Second, this interaction isn't direct but goes through a high level of abstraction: dancer's emotional expression is recognized and is the source of music generation. Third, this interaction has been designed and validated through constant collaboration with a choreographer, culminating in an augmented ballet performance in front of a live audience.

Mots clés: Interactive sonification; motion; gesture and music; interaction; live performance; musical human-computer interaction

Domaines: Interface homme-machine [cs.HC]; Son [cs.SD]

Alexis Clay, Nadine Couture, Elodie Decarsin, Myriam Desainte-Catherine, Pierre-Henri Vulliard, et al.. Movement to emotions to music: using whole body emotional expression as an interaction for electronic music generation. New Interfaces for Musical Expression, May 2012, Ann Arbor, United States. ⟨hal-00771385⟩