Spatiotemporal MRF approach to video segmentation: Application to motion detection and lip segmentation

Franck Luthon (1) , Alice Caplier (1) , Marc Liévin (1)

1 LIS - Laboratoire des images et des signaux

In this paper, a spatiotemporal strategy for image sequence analysis is proposed: a video sequence is processed as a 3-D data batch instead of a series of 2-D images. Applying this approach to motion detection, a 3-D Markovian model associated with a spatiotemporal relaxation is defined. Using a 3-D neighbourhood of pixels for modelling spatiotemporal interactions, robust results are obtained for detecting moving objects in noisy sequences or in the case of overlapping motion. In order to improve the performance to detect poorly-textured objects or very slow motion, the algorithm is integrated in a spatiotemporal multiresolution scheme. The data pyramid is built by using 3-D low-pass filtering and 3-D subsampling. Robust results for synthetic and real-world outdoor image sequences are reported. This approach is also applied successfully to speaker's lip segmentation in image sequences, for audiovisual telecommunication.

Mots clés: motion detection; image sequences; Markov Random Field (MRF); spatiotemporal approach; multiresolution; lip segmentation

Domaines: Traitement du signal et de l'image [eess.SP]; Traitement du signal et de l'image [eess.SP]

Franck Luthon, Alice Caplier, Marc Liévin. Spatiotemporal MRF approach to video segmentation: Application to motion detection and lip segmentation. Signal Processing, 1999, 76 (1), pp.61-80. ⟨hal-00785993⟩