Learning person-object interactions for action recognition in still images

V. Delaitre (1) , J. Sivic (1) , I. Laptev (1)

1 WILLOW - Models of visual object recognition and scene understanding

We investigate a discriminatively trained model of person-object interactions for recognizing common human actions in still images. We build on the locally order-less spatial pyramid bag-of-features model, which was shown to perform extremely well on a range of object, scene and human action recognition tasks. We introduce three principal contributions. First, we replace the standard quantized local HOG/SIFT features with stronger discriminatively trained body part and object detectors. Second, we introduce new person-object interaction features based on spatial co-occurrences of individual body parts and objects. Third, we address the combinatorial problem of a large number of possible interaction pairs and propose a discriminative selection procedure using a linear support vector machine (SVM) with a sparsity inducing regularizer. Learning of action-specific body part and object interactions bypasses the difficult problem of estimating the complete human body pose configuration. Benefits of the proposed model are shown on human action recognition in consumer photographs, outperforming the strong bag-of-features baseline.

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]

V. Delaitre, J. Sivic, I. Laptev. Learning person-object interactions for action recognition in still images. NIPS 2011 : Twenty-Fifth Annual Conference on Neural Information Processing Systems, NIPS Foundation, Dec 2011, Grenada, Spain. ⟨hal-00648156⟩