Nonlinear Color Space and Spatiotemporal MRF for Hierarchical Segmentation of Face Features in Video

Marc Liévin (1) , Franck Luthon (2)

1 LIS - Laboratoire des images et des signaux
2 LIUPPA - Laboratoire Informatique de l'Université de Pau et des Pays de l'Adour

This paper addresses the design of the image processing stage for face analysis. In human computer interfaces, user-friendliness requires robustness and good quality in image processing. To cope with unsupervised lighting conditions and unknown speaker, two original preprocessing tools are introduced here: a logarithmic color transform and an entropy-based motion threshold. As regards the main processing stage, a hierarchical segmentation scheme based on Markov random fields is proposed, that combines color and motion observations within a spatiotemporal neighborhood. Relevant face regions are thereafter automatically segmented. The good quality of the label fields enables localization and tracking of the face. Geometrical measurements on facial feature edges, such as lips or eyes, are provided by an active contour postprocessing stage. Results are shown both on well-known test sequences, and also on typical sequences acquired from micro and motorized cameras. The robustness and accuracy of the extracted contours are promising for any real-time application aiming at facial communication in unsupervised viewing conditions.

Mots clés: entropy power; Facial analysis; face feature; liptracking; segmentation; Markov random field; hue; motion; entropy power.

Domaines: Traitement du signal et de l'image [eess.SP]; Traitement du signal et de l'image [eess.SP]

Marc Liévin, Franck Luthon. Nonlinear Color Space and Spatiotemporal MRF for Hierarchical Segmentation of Face Features in Video. IEEE Transactions on Image Processing, 2004, 13 (1), pp.63-71. ⟨10.1109/TIP.2003.818013⟩. ⟨hal-00785936⟩