GMM Mapping Of Visual Features of Cued Speech From Speech Spectral Features

Zuheng Ming (1) , Denis Beautemps (2) , Gang Feng (2)

1 Service d'ORL et de chirurgie cervicale
2 GIPSA-MAGIC - GIPSA - Machines parlantes, Gestes oro-faciaux, Interaction Face-à-face, Communication augmentée

In this paper, we present a statistical method based on GMM modeling to map the acoustic speech spectral features to visual features of Cued Speech in the regression criterion of Minimum Mean-Square Error (MMSE) in a low signal level which is innovative and different with the classic text-to-visual approach. Two different training methods for GMM, namely Expectation-Maximization (EM) approach and supervised training method were discussed respectively. In comparison with the GMM based mapping modeling we first present the results with the use of a Multiple-Linear Regression (MLR) model also at the low signal level and study the limitation of the approach. The experimental results demonstrate that the GMM based mapping method can significantly improve the mapping performance compared with the MLR mapping model especially in the sense of the weak linear correlation between the target and the predictor such as the hand positions of Cued Speech and the acoustic speech spectral features.

Mots clés: GMM mapping; MFCC; LSP; Cued Speech

Domaines: Sciences de l'information et de la communication

Zuheng Ming, Denis Beautemps, Gang Feng. GMM Mapping Of Visual Features of Cued Speech From Speech Spectral Features. AVSP 2013 - 12th International Conference on Auditory-Visual Speech Processing, Aug 2013, Annecy, France. pp.191 - 196. ⟨hal-00863875⟩