Motion-Sound Mapping through Interaction - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue ACM Transactions on Interactive Intelligent Systems Année : 2018

Motion-Sound Mapping through Interaction

Résumé

Technologies for sensing movement are expanding toward everyday use in virtual reality, gaming, and artistic practices. In this context, there is a need for methodologies to help designers and users create meaningful movement experiences. This article discusses a user-centered approach for the design of interactive auditory feedback using interactive machine learning. We discuss Mapping through Interaction, a method for crafting sonic interactions from corporeal demonstrations of embodied associations between motion and sound. It uses an interactive machine learning approach to build the mapping from user demonstrations, emphasizing an iterative design process that integrates acted and interactive experiences of the relationships between movement and sound. We examine Gaussian Mixture Regression and Hidden Markov Regression for continuous movement recognition and real-time sound parameter generation. We illustrate and evaluate this approach through an application in which novice users can create interactive sound feedback based on coproduced gestures and vocalizations. Results indicate that Gaussian Mixture Regression and Hidden Markov Regression can efficiently learn complex motion-sound mappings from few examples.
Fichier principal
Vignette du fichier
TiiS_HCML___Mapping_through_Interaction.pdf (2.18 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-02409300 , version 1 (06-09-2021)

Identifiants

Citer

Jules Françoise, Frederic Bevilacqua. Motion-Sound Mapping through Interaction: An Approach to User-Centered Design of Auditory Feedback Using Machine Learning. ACM Transactions on Interactive Intelligent Systems , 2018, 8 (2), pp.16. ⟨10.1145/3211826⟩. ⟨hal-02409300⟩
141 Consultations
266 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More