Visual Focus of Attention estimation with unsupervised incremental learning

Stefan Duffner 1 Christophe Garcia 1
1 imagine - Extraction de Caractéristiques et Identification
LIRIS - Laboratoire d'InfoRmatique en Image et Systèmes d'information
Abstract : In this paper, we propose a new method for estimating the Visual Focus Of Attention (VFOA) in a video stream captured by a single distant camera and showing several persons sitting around table, like in formal meeting or video-conferencing settings. The visual targets for a given person are automatically extracted on-line using an unsupervised algorithm that incrementally learns the different appearance clusters from low-level visual features computed from face patches provided by a face tracker without the need of an intermediate error-prone step of head-pose estimation as in classical approaches. The clusters learnt in that way can then be used to classify the different visual attention targets of the person during a tracking run, without any prior knowledge on the environment and the configuration of the room or the visible persons. Experiments on public datasets containing almost two hours of annotated videos from meetings and video-conferencing show that the proposed algorithm produces state-of-the-art results and even outperforms a traditional supervised method that is based on head orientation estimation and that classifies visual focus of attention using Gaussian Mixture Models.
Complete list of metadatas

Cited literature [40 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01153969
Contributor : Stefan Duffner <>
Submitted on : Wednesday, May 20, 2015 - 5:59:07 PM
Last modification on : Thursday, November 21, 2019 - 2:28:59 AM
Long-term archiving on: Thursday, April 20, 2017 - 5:53:39 AM

File

tcsvt2015_duffner.pdf
Files produced by the author(s)

Identifiers

Citation

Stefan Duffner, Christophe Garcia. Visual Focus of Attention estimation with unsupervised incremental learning. IEEE Transactions on Circuits and Systems for Video Technology, Institute of Electrical and Electronics Engineers, 2015, ⟨10.1109/TCSVT.2015.2501920⟩. ⟨hal-01153969⟩

Share

Metrics

Record views

455

Files downloads

642