Animation of generic 3D Head models driven by speech

Lucas Terissi (1) , Mauricio Cerda (2) , Juan C. Gomez (1) , Nancy Hitschfeld-Kahler (3) , Bernard Girau (2) , Renato Valenzuela (3)

1 Laboratory for System Dynamics and Signal Processing
2 CORTEX - Neuromimetic intelligence
3 DCC - Computer Science Department [Santiago]

In this paper, a system for speech-driven animation of generic 3D head models is presented. The system is based on the inversion of a joint Audio-Visual Hidden Markov Model to estimate the visual information from speech data. Estimated visual speech features are used to animate a simple face model. The animation of a more complex head model is then obtained by automatically mapping the deformation of the simple model to it. The proposed algorithm allows the animation of 3D head models of arbitrary complexity through a simple setup procedure. The resulting animation is evaluated in terms of intelligibility of visual speech through subjective tests, showing a promising performance.

Mots clés: Facial Animation; Hidden Markov Models; Audio-Visual Speech Processing

Domaines: Synthèse d'image et réalité virtuelle [cs.GR]; Multimédia [cs.MM]; Géométrie algorithmique [cs.CG]

Lucas Terissi, Mauricio Cerda, Juan C. Gomez, Nancy Hitschfeld-Kahler, Bernard Girau, et al.. Animation of generic 3D Head models driven by speech. IEEE International Conference on Multimedia and Expo - ICME 2011, Jul 2011, Barcelone, Spain. To appear. ⟨hal-00587016⟩