Toward a multi-speaker visual articulatory feedback system

Atef Ben Youssef (1) , Thomas Hueber (1) , Pierre Badin (1) , Gérard Bailly (1)

1 GIPSA-MAGIC - GIPSA - Machines parlantes, Gestes oro-faciaux, Interaction Face-à-face, Communication augmentée

In this paper, we present recent developments on the HMMbased acoustic-to-articulatory inversion approach that we develop for a "visual articulatory feedback" system. In this approach, multi-stream phoneme HMMs are trained jointly on synchronous streams of acoustic and articulatory data, acquired by electromagnetic articulography (EMA). Acousticto- articulatory inversion is achieved in two steps. Phonetic and state decoding is first performed. Then articulatory trajectories are inferred from the decoded phone and state sequence using the maximum-likelihood parameter generation algorithm (MLPG). We introduce here a new procedure for the reestimation of the HMM parameters, based on the Minimum Generation Error criterion (MGE). We also investigate the use of model adaptation techniques based on maximum likelihood linear regression (MLLR), as a first step toward a multispeaker visual articulatory feedback system.

Mots clés: Acoustic-articulatory inversion; ElectroMagnetic Articulography (EMA); Hidden Markov Model (HMM); Minimum Generation Error (MGE); Speaker adaptation; Maximum Likelihood Linear Regression (MLLR)

Domaines: Traitement du signal et de l'image [eess.SP]; Traitement du signal et de l'image [eess.SP]

Atef Ben Youssef, Thomas Hueber, Pierre Badin, Gérard Bailly. Toward a multi-speaker visual articulatory feedback system. Interspeech 2011 - 12th Annual Conference of the International Speech Communication Association, Aug 2011, Florence, Italy. pp.589-592. ⟨hal-00618781⟩