Reactive Statistical Mapping: Towards the Sketching of Performative Control with Data

Abstract : This paper presents the results of our participation to the ninth eNTERFACE workshop on multimodal user interfaces. Our target for this workshop was to bring some technologies currently used in speech recognition and synthesis to a new level, i.e. being the core of a new HMM-based mapping system. The idea of statistical mapping has been investigated, more precisely how to use Gaussian Mixture Models and Hidden Markov Models for realtime and reactive generation of new trajectories from inputted labels and for realtime regression in a continuous-to-continuous use case. As a result, we have developed several proofs of concept, including an incremental speech synthesiser, a software for exploring stylistic spaces for gait and facial motion in realtime, a reactive audiovisual laughter and a prototype demonstrating the realtime reconstruction of lower body gait motion strictly from upper body motion, with conservation of the stylistic properties. This project has been the opportunity to formalise HMM-based mapping, integrate various of these innovations into the Mage library and explore the development of a realtime gesture recognition tool.
Document type :
Conference papers
Complete list of metadatas

Cited literature [40 references]  Display  Hide  Download
Contributor : Adela Barbulescu <>
Submitted on : Monday, August 1, 2016 - 3:56:10 PM
Last modification on : Monday, April 30, 2018 - 3:02:01 PM
Long-term archiving on : Tuesday, November 8, 2016 - 7:47:57 PM


Files produced by the author(s)



Nicolas D’alessandro, Joëlle Tilmanne, Maria Astrinaki, Thomas Hueber, Rasmus Dall, et al.. Reactive Statistical Mapping: Towards the Sketching of Performative Control with Data. 9th International Summer Workshop on Multimodal Interfaces (eNTERFACE), Jul 2013, Lisbonne, Portugal. pp.20-49, ⟨10.1007/978-3-642-55143-7_2⟩. ⟨hal-01134574⟩



Record views


Files downloads