Automatic dynamic template tracking of inner lips based on CLNF

Li Liu (1) , Gang Feng (2) , Denis Beautemps (1)

1 GIPSA-CRISSP - GIPSA - Cognitive Robotics, Interactive Systems, & Speech Processing
2 GIPSA-VSLD - GIPSA - Voix Systèmes Linguistiques et Dialectologie

In this paper, a novel automatic approach to extract the inner lips contour of speakers without using artifices is proposed. This method is based on a recent facial contour extraction model developed in computer vision, called Constrained Local Neural Field (CLNF), which provides 8 characteristic points (landmarks) defining the inner lips contour. However, directly applied to our visual data including Cued Speech (CS) data, CLNF failed in about 50% of cases. We propose a Modified CLNF to estimate inner lips contour based on original CLNF landmarks. A dynamic template using the first derivative of smoothed luminance variation is explored in this new model. This method gives precise estimation of aperture for inner lips. It is evaluated on 4800 images of three French speakers. The proposed method corrects 95% CLNF errors and total RMSE of one pixel (i.e. 0.05cm in average) is reached, instead of four pixels using original CLNF.

Mots clés: Cued Speech; inner lips contour parameter; dynamic template; luminance variation; CLNF

Domaines: Informatique; Sciences de l'Homme et Société; Informatique [cs]

Li Liu, Gang Feng, Denis Beautemps. Automatic dynamic template tracking of inner lips based on CLNF. ICASSP 2017 - IEEE International Conference on Acoustics, Speech and Signal Processing, Mar 2017, New Orleans, United States. ⟨hal-01504342⟩