Automatic dynamic template tracking of inner lips based on CLNF

Li Liu 1 Gang Feng 2 Denis Beautemps 1
1 GIPSA-CRISSP - CRISSP
GIPSA-DPC - Département Parole et Cognition
2 GIPSA-VSLD - VSLD
GIPSA-DPC - Département Parole et Cognition
Abstract : In this paper, a novel automatic approach to extract the inner lips contour of speakers without using artifices is proposed. This method is based on a recent facial contour extraction model developed in computer vision, called Constrained Local Neural Field (CLNF), which provides 8 characteristic points (landmarks) defining the inner lips contour. However, directly applied to our visual data including Cued Speech (CS) data, CLNF failed in about 50% of cases. We propose a Modified CLNF to estimate inner lips contour based on original CLNF landmarks. A dynamic template using the first derivative of smoothed luminance variation is explored in this new model. This method gives precise estimation of aperture for inner lips. It is evaluated on 4800 images of three French speakers. The proposed method corrects 95% CLNF errors and total RMSE of one pixel (i.e. 0.05cm in average) is reached, instead of four pixels using original CLNF.
Complete list of metadatas

Cited literature [12 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01504342
Contributor : Li Liu <>
Submitted on : Sunday, April 9, 2017 - 9:03:57 PM
Last modification on : Friday, July 27, 2018 - 11:04:38 AM
Long-term archiving on : Monday, July 10, 2017 - 12:20:13 PM

File

LIU.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01504342, version 1

Collections

Citation

Li Liu, Gang Feng, Denis Beautemps. Automatic dynamic template tracking of inner lips based on CLNF. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2017), Mar 2017, New Orleans, United States. ⟨hal-01504342⟩

Share

Metrics

Record views

360

Files downloads

235