Compressed Fisher Vectors for Large-Scale Image Classification

Jorge Sanchez (1) , Florent Perronnin (2) , Thomas Mensink (2, 3) , Jakob Verbeek (3)

1 FaMAF - Facultad de Matemática, Astronomía y Física [Cordoba]
2 Xerox Research Centre Europe [Meylan]
3 LEAR - Learning and recognition in vision

A standard approach to describe an image for image classification and image retrieval, is to extract a set of local patch descriptors, encode them into a high dimensional vector and pool them into an image-level signature. The most common patch encoding strategy consists in quantizing the local descriptors into a finite set of prototypical elements. This leads to the popular Bag-of-Visual words (BOV) representation. In this work, we propose to use the Fisher Kernel framework as an alternative patch encoding strategy: we describe patches by their deviation from an ''universal'' generative model. This representation, which we call Fisher Vector (FV) has many advantages: it is efficient to compute, it leads to excellent results even with costless linear classifiers, and it can be compressed with a minimal loss of accuracy using product quantization. We report experimental results on five standard datasets -- PASCAL VOC 2007, Caltech 256, SUN 397, ILSVRC 2010 and ImageNet10K -- with up to 9M images and 10K classes, showing state-of-the-art results with the FV framework.

Domaines: Apprentissage [cs.LG]

Jorge Sanchez, Florent Perronnin, Thomas Mensink, Jakob Verbeek. Compressed Fisher Vectors for Large-Scale Image Classification. [Research Report] RR-8209, 2013. ⟨hal-00779493v1⟩