Gesture-Sound Mapping by Demonstration in Interactive Music Systems

Jules Françoise (1)

1 Interactions Son Musique Mouvement

In this paper we address the issue of mapping between gesture and sound in interactive music systems. Our approach, we call mapping by demonstration, aims at learning the mapping from examples provided by users while interacting with the system. We propose a general framework for modeling gesture-sound sequences based on a probabilistic, multimodal and hierarchical model. Two orthogonal modeling aspects are detailed and we describe planned research directions to improve and evaluate the proposed models.

Mots clés: gesture; hierarchical modeling; hmm; mapping; multimodal; music performance; sound synthesis

Domaines: Son [cs.SD]

Jules Françoise. Gesture-Sound Mapping by Demonstration in Interactive Music Systems. Proceedings of the 21st ACM international conference on Multimedia (MM'13), Oct 2013, Barcelona, Spain, France. pp.1051-1054, ⟨10.1145/2502081.2502214⟩. ⟨hal-01061221⟩