An audiovisual attention model for natural conversation scenes

Antoine Coutrot (1) , Nathalie Guyader (1)

1 GIPSA-VIBS - GIPSA - Vision and Brain Signal Processing

Classical visual attention models neither consider social cues, such as faces, nor auditory cues, such as speech. However, faces are known to capture visual attention more than any other visual features, and recent studies showed that speech turn-taking affects the gaze of non-involved viewers. In this paper, we propose an audiovisual saliency model able to predict the eye movements of observers viewing other people having a conversation. Thanks to a speaker diarization algorithm, our audiovisual saliency model increases the saliency of the speakers compared to the addressees. We evaluated our model with eye-tracking data, and found that it significantly outperforms visual attention models using an equal and constant saliency value for all faces.

Mots clés: audiovisual saliency model; eye movements; speaker diarization; speech; social gaze

Domaines: Sciences cognitives; Neurosciences; Traitement du signal et de l'image [eess.SP]; Traitement du signal et de l'image [eess.SP]; Psychologie

Antoine Coutrot, Nathalie Guyader. An audiovisual attention model for natural conversation scenes. ICIP 2014 - 21st IEEE International Conference on Image Processing, Oct 2014, Paris, France. pp.1-5. ⟨hal-01009467⟩