Multimodality Acquisition of Articulatory Data and Processing

Michael Aron (1) , Anastasios Roussos (2) , Marie-Odile Berger (1) , Erwan Kerrien (1) , Petros Maragos (2)

1 MAGRIT - Visual Augmentation of Complex Environments
2 Computer Vision, Speech Communication and Signal Processing Group

In this paper, a framework to acquire and process dynamic data of the tongue during speech processing is presented. First, a setup to acquire data of the tongue shape combining ultrasound images, electromagnetic localization sensors and sound is presented. Techniques to automatically calibrate and synchronize the data are described. A method to extract the tongue shape is then proposed, by combining a preprocessing of the ultrasound images with an imagebased tracking method that integrates adapted constraints.

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]; Traitement du signal et de l'image [eess.SP]; Traitement du signal et de l'image [eess.SP]

Michael Aron, Anastasios Roussos, Marie-Odile Berger, Erwan Kerrien, Petros Maragos. Multimodality Acquisition of Articulatory Data and Processing. 16th European Signal Processing Conference - EUSIPCO 2008, EPFL, Aug 2008, Lausanne, Switzerland. ⟨inria-00316041⟩