How different kinds of sound in videos can influence gaze

Guanghan Song (1) , Denis Pellerin (1) , Lionel Granjon (2)

1 GIPSA-AGPIG - GIPSA - Architecture, Géométrie, Perception, Images, Gestes
2 GIPSA-Services - GIPSA-Services

This paper presents an analysis of the effect of thirteen different kinds of sound on visual gaze when looking freely at videos to help to predict eye positions. First, an audio-visual experiment was designed with two groups of participants, with audio-visual (AV) and visual (V) conditions, to test the sound effect. Then, an audio experiment was designed to validate the classification of sound we proposed. We observed that the sound effect is different depending on the kind of sound, and that the classes with human voice (speech, singer, human noise and singers) have the greatest effect. Finally, a comparison of eye positions with a visual saliency model was carried out, which proves that adding sound to video decreases the accuracy of prediction of the visual saliency model.

Mots clés: Sound; Audio-visual experiment; Visual gaze; Visual saliency; Video

Domaines: Traitement du signal et de l'image [eess.SP]; Traitement du signal et de l'image [eess.SP]

Guanghan Song, Denis Pellerin, Lionel Granjon. How different kinds of sound in videos can influence gaze. WIAMIS 2012 - 13th International Workshop on Image Analysis for Multimedia Interactive Services, May 2012, Dublin, Ireland. 4 p., ⟨10.1109/WIAMIS.2012.6226776⟩. ⟨hal-00734135⟩