Towards Understanding and Modeling Audiovisual Saliency Based on Talking Faces

Naty Ould-Sidaty 1 Mohamed-Chaker Larabi 1 Abdelhakim Saadane 1
1 XLIM-SIC - SIC
Université de Poitiers, XLIM - XLIM
Abstract : Usual attention is an important mechanism of the human visual system. It allows reducing the amount of information to be processed and accelerates the overall process of vision. Several models for images and videos have been proposed in the literature with encouraging results. However, most existing saliency models do not take into account the multimodal aspect of the video (audio and image). In this paper, we propose to investigate the influence of audio on visual attention. From one side, we carried out eye-tracking experiments for recording subjects' eye movement when watching videos. From another side, eye positions (fixation duration) were used for computing and comparing video attention maps (through eye-movements) with and without audio using state-of-the-art measures. Results therefore showed that audio significantly affect the viewer's attention and consequently it must be taken into consideration for the development of any multimedia saliency model. The findings of these experiments have been used for the development of audiovisual saliency model based on talking face. Result obtained with our model were assessed using usual measurements and showed good performance with regards to ground truth.
Type de document :
Communication dans un congrès
Tenth International Conference on Signal-Image Technology and Internet-Based Systems (SITIS), Nov 2014, Marrakech, Morocco. pp.508 - 515, 2014, <10.1109/SITIS.2014.110>
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01159122
Contributeur : Mohamed-Chaker Larabi <>
Soumis le : mardi 2 juin 2015 - 16:03:54
Dernière modification le : mercredi 20 janvier 2016 - 17:54:20

Identifiants

Collections

Citation

Naty Ould-Sidaty, Mohamed-Chaker Larabi, Abdelhakim Saadane. Towards Understanding and Modeling Audiovisual Saliency Based on Talking Faces. Tenth International Conference on Signal-Image Technology and Internet-Based Systems (SITIS), Nov 2014, Marrakech, Morocco. pp.508 - 515, 2014, <10.1109/SITIS.2014.110>. <hal-01159122>

Partager

Métriques

Consultations de la notice

178