Estimation of Speech Lip Features from Discrete Cosinus Transform

Zuheng Ming (1) , Denis Beautemps (2) , Gang Feng (2) , Sébastien A. Schmerber (1)

1 Service d'ORL et de chirurgie cervicale
2 GIPSA-MAGIC - GIPSA - Machines parlantes, Gestes oro-faciaux, Interaction Face-à-face, Communication augmentée

This study is a contribution to the field of visual speech processing. It focuses on the automatic extraction of Speech lip features from natural lips. The method is based on the direct prediction of these features from predictors derived from an adequate transformation of the pixels of the lip region of interest. The transformation is made of a 2-D Discrete Cosine Transform combined with a Principal Component Analysis applied to a subset of the DCT coefficients corresponding to about 1% of the total DCTs. The results show the possibility to estimate the geometric lip feature with a good accuracy (a root mean square of 1 to 1.4 mm for the lip aperture and the lip width) using a reduce set of predictors derived from the PCA.

Domaines: Sciences de l'information et de la communication

Zuheng Ming, Denis Beautemps, Gang Feng, Sébastien A. Schmerber. Estimation of Speech Lip Features from Discrete Cosinus Transform. Interspeech 2010 - 11th Annual Conference of the International Speech Communication Association, Sep 2010, Makuhari, Japan. pp.1612 - 1615. ⟨hal-00536131⟩