A tensor motion descriptor based on histograms of gradients and optical flow

Virginia Fernandes Mota (1) , Eder de Almeida Perez (1) , Silva Maciel Luiz Maurílio Da (1) , Marcelo Bernardes Vieira (1) , Philippe-Henri Gosselin (2, 3)

1 DCC / ICE - UFJF - Departamento de Ciência da Computação [Juiz de Fora]
2 TEXMEX - Multimedia content-based indexing
3 MIDI - Multimedia Indexation and Data Integration

This paper presents a new tensor motion descriptor only using optical flow and HOG3D information: no interest points are extracted and it is not based on a visual dictionary. We propose a new aggregation technique based on tensors. This is a double aggregation of tensor descriptors. The first one represents motion by using polynomial coefficients which approximates the optical flow. The other represents the accumulated data of all histograms of gradients of the video. The descriptor is evaluated by a classification of KTH, UCF11 and Hollywood2 datasets, using a SVM classifier. Our method reaches 93.2% of recognition rate with KTH, comparable to the best local ap- proaches. For the UCF11 and Hollywood2 datasets, our recognition achieves fairly competitive results compared to local and learning based approaches. Keywords: Global motion descriptor, optical flow, histogram of gradients, action recognition

Domaines: Machine Learning [stat.ML]; Vision par ordinateur et reconnaissance de formes [cs.CV]

Virginia Fernandes Mota, Eder de Almeida Perez, Silva Maciel Luiz Maurílio Da, Marcelo Bernardes Vieira, Philippe-Henri Gosselin. A tensor motion descriptor based on histograms of gradients and optical flow. Pattern Recognition Letters, 2014, 39, pp.85-91. ⟨10.1016/j.patrec.2013.08.008⟩. ⟨hal-00861395⟩