Fine-Grained Action Detection and Classification in Table Tennis with Siamese Spatio-Temporal Convolutional Neural Network

Pierre-Etienne Martin (1, 2) , Jenny Benois-Pineau (1) , Renaud Péteri (3)

1 LaBRI - Laboratoire Bordelais de Recherche en Informatique
2 UB - Université de Bordeaux
3 MIA - Mathématiques, Image et Applications - EA 3165

Human action recognition in videos is one of the key problems in visual data interpretation. Despite intensive research, the recognition of actions with low inter-class variability remains a challenge. To answer this problem, my thesis focus on fine-grained classification challenge using a Siamese Spatio-Temporal Convolutional Neural Network and apply it to a new dataset we have introduced TTStroke-21. Our model take as input data RGB images and Optical Flow and is able to reach an accuracy of 91.4% against 43.1% for our baseline on temporal segmented videos. Detection and classification in videos using a sliding temporal window leads to a score of 81.3% over the whole dataset.

Mots clés: Deep Learning; Optical Flow; Action classification; Spatio-temporal convolution

Domaines: Multimédia [cs.MM]

Pierre-Etienne Martin, Jenny Benois-Pineau, Renaud Péteri. Fine-Grained Action Detection and Classification in Table Tennis with Siamese Spatio-Temporal Convolutional Neural Network. 2019 IEEE International Conference on Image Processing (ICIP), Sep 2019, Taipei, Taiwan. pp.3027-3028, ⟨10.1109/ICIP.2019.8803382⟩. ⟨hal-02326229⟩