SkeleMotion: A New Representation of Skeleton Joint Sequences Based on Motion Information for 3D Action Recognition

Carlos Caetano (1) , Jessica Sena (1) , Francois F Bremond (2) , Jefersson Alex dos Santos (3) , William Robson Schwartz (1)

1 UFMG - Universidade Federal de Minas Gerais = Federal University of Minas Gerais [Belo Horizonte, Brazil]
2 STARS - Spatio-Temporal Activity Recognition Systems
3 RECOD - REasoning for Complex Data

Due to the availability of large-scale skeleton datasets, 3D human action recognition has recently called the attention of computer vision community. Many works have fo-cused on encoding skeleton data as skeleton image representations based on spatial structure of the skeleton joints, in which the temporal dynamics of the sequence is encoded as variations in columns and the spatial structure of each frame is represented as rows of a matrix. To further improve such representations, we introduce a novel skeleton image representation to be used as input of Convolutional Neural Networks (CNNs), named SkeleMotion. The proposed approach encodes the temporal dynamics by explicitly computing the magnitude and orientation values of the skeleton joints. Different temporal scales are employed to compute motion values to aggregate more temporal dynamics to the representation making it able to capture long-range joint interactions involved in actions as well as filtering noisy motion values. Experimental results demonstrate the effectiveness of the proposed representation on 3D action recognition outperforming the state-of-the-art on NTU RGB+D 120 dataset.

Domaines: Traitement des images [eess.IV]

Carlos Caetano, Jessica Sena, Francois F Bremond, Jefersson Alex dos Santos, William Robson Schwartz. SkeleMotion: A New Representation of Skeleton Joint Sequences Based on Motion Information for 3D Action Recognition. AVSS 2019 - 16th IEEE International Conference on Advanced Video and Signal-based Surveillance, Sep 2019, Taipei, Taiwan. ⟨hal-02422551⟩