Speaker-Adaptive Acoustic-Articulatory Inversion using Cascaded Gaussian Mixture Regression

Thomas Hueber 1 Laurent Girin 1 Xavier Alameda-Pineda 2 Gérard Bailly 1
1 GIPSA-CRISSP - CRISSP
GIPSA-DPC - Département Parole et Cognition
2 PERCEPTION - Interpretation and Modelling of Images and Videos
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, INPG - Institut National Polytechnique de Grenoble
Abstract : This paper addresses the adaptation of an acoustic-articulatory model of a reference speaker to the voice of another speaker, using a limited amount of audio-only data. In the context of pronunciation training, a virtual talking head displaying the internal speech articulators (e.g., the tongue) could be automatically animated by means of such a model using only the speaker's voice. In this study, the articulatory-acoustic relationship of the reference speaker is modeled by a gaussian mixture model (GMM). To address the speaker adaptation problem, we propose a new framework called cascaded Gaussian mixture regression (C-GMR), and derive two implementations. The first one, referred to as Split-C-GMR, is a straightforward chaining of two distinct GMRs: one mapping the acoustic features of the source speaker into the acoustic space of the reference speaker, and the other estimating the articulatory trajectories with the reference model. In the second implementation, referred to as Integrated-C-GMR, the two mapping steps are tied together in a single probabilistic model. For this latter model, we present the full derivation of the exact EM training algorithm, that explicitly exploits the missing data methodology of machine learning. Other adaptation schemes based on maximum-a posteriori (MAP), maximum likelihood linear regression (MLLR) and direct cross-speaker acoustic-to-articulatory GMR are also investigated. Experiments conducted on two speakers for different amount of adaptation data show the interest of the proposed C-GMR techniques.
Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-01231197
Contributor : Thomas Hueber <>
Submitted on : Thursday, November 19, 2015 - 4:25:12 PM
Last modification on : Wednesday, April 11, 2018 - 1:58:54 AM

Identifiers

Citation

Thomas Hueber, Laurent Girin, Xavier Alameda-Pineda, Gérard Bailly. Speaker-Adaptive Acoustic-Articulatory Inversion using Cascaded Gaussian Mixture Regression. IEEE/ACM Transactions on Audio, Speech and Language Processing, Institute of Electrical and Electronics Engineers, 2015, 23 (12), pp.2246-2259. ⟨http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7180325⟩. ⟨10.1109/TASLP.2015.2464702⟩. ⟨hal-01231197⟩

Share

Metrics

Record views

535