Skip to Main content Skip to Navigation
Journal articles

Modelling spatio-temporal saliency to predict gaze direction for short videos

Sophie Marat 1 Tien Ho Phuoc 1 Lionel Granjon 2 Nathalie Guyader 1 Denis Pellerin 1 Anne Guérin-Dugué 1
1 GIPSA-GPIG - GIPSA - Géométrie, Perception, Images, Geste
GIPSA-DIS - Département Images et Signal
2 GIPSA-Services - GIPSA-Services
GIPSA-lab - Grenoble Images Parole Signal Automatique
Abstract : This paper presents a spatio-temporal saliency model that predicts eye movement during video free viewing. This model is inspired by the biology of the first steps of the human visual system. The model extracts two signals from video stream corresponding to the two main outputs of the retina: parvocellular and magnocellular. Then, both signals are split into elementary feature maps by cortical-like filters. These feature maps are used to form two saliency maps: a static and a dynamic one. These maps are then fused into a spatio-temporal saliency map. The model is evaluated by comparing the salient areas of each frame predicted by the spatio-temporal saliency map to the eye positions of different subjects during a free video viewing experiment with a large database (17000 frames). In parallel, the static and the dynamic pathways are analyzed to understand what is more or less salient and for what type of videos our model is a good or a poor predictor of eye movement.
Complete list of metadata

Cited literature [35 references]  Display  Hide  Download
Contributor : Denis Pellerin Connect in order to contact the contributor
Submitted on : Monday, March 16, 2009 - 4:30:57 PM
Last modification on : Tuesday, October 19, 2021 - 11:22:42 PM
Long-term archiving on: : Tuesday, June 8, 2010 - 11:31:22 PM


Files produced by the author(s)



Sophie Marat, Tien Ho Phuoc, Lionel Granjon, Nathalie Guyader, Denis Pellerin, et al.. Modelling spatio-temporal saliency to predict gaze direction for short videos. International Journal of Computer Vision, Springer Verlag, 2009, 82 (3), pp.231-243. ⟨10.1007/s11263-009-0215-3⟩. ⟨hal-00368496⟩



Les métriques sont temporairement indisponibles