Semantic Contexts and Fisher Vectors for the ImageCLEF 2011 Photo Annotation Task

Yu Su (1) , Frédéric Jurie (1)

1 Equipe Image - Laboratoire GREYC - UMR6072

This paper describes the participation of UNICAEN/ GREYC to the ImageCLEF 2011 photo annotation task. The proposed approach uses visual image features and binary annotations of concepts only. In this approach, the annotations are predicted by SVM classi ers trained separately for each concept. The classi ers take Bag-of-Words histograms and sher vectors representations as inputs, both being combined at the decision level. Furthermore, contextual information is also embedded into the Bag-of-Words histograms to enhance their performance. The experimental results show that the combination of Bag-of-Words histograms and Fisher vectors brings signi cant performance increase (e.g. 4% for Mean Average Precision). Furthermore, the results of our best-run rank in top 3 for both concept and image level evaluations.

Mots clés: Image classi cation; Photo annotation; Bag-of-Words model; Semantic context; Fisher Vectors

Domaines: Traitement des images [eess.IV]

Yu Su, Frédéric Jurie. Semantic Contexts and Fisher Vectors for the ImageCLEF 2011 Photo Annotation Task. CLEF (Notebook Papers/Labs/Workshop), 2011, Amsterdam, Netherlands. 11 p. ⟨hal-00808656⟩