Sport Action Recognition with Siamese Spatio-Temporal CNNs: Application to Table Tennis

Pierre-Etienne Martin (1, 2) , Jenny Benois-Pineau (1) , Renaud Péteri (3) , Julien Morlier (4)

1 LaBRI - Laboratoire Bordelais de Recherche en Informatique
2 UB - Université de Bordeaux
3 MIA - Mathématiques, Image et Applications - EA 3165
4 IMS - Laboratoire de l'intégration, du matériau au système

Human action recognition in video is one of the key problems in visual data interpretation. Despite intensive research, the recognition of actions with low inter-class variability remains a challenge. This paper presents a new Siamese Spatio-Temporal Convolutional neural network (SSTC) for this purpose. When applied to table tennis, it is possible to detect and recognize 20 table tennis strokes. The model has been trained on a specific dataset, TTStroke-21, recorded in natural condition (markerless) at the Faculty of Sports of the University of Bordeaux. Our model takes as inputs a RGB image sequence and its computed Optical Flow. After 3 spatio-temporal convolutions, data are fused in a fully connected layer of a proposed siamese network architecture. Our method reaches an accuracy of 91.4% against 43.1% for our baseline.

Mots clés: sport video analysis; spatio-temporal convolu- tions; Siamese neural network; Index Terms-Action recognition

Domaines: Informatique [cs]

Pierre-Etienne Martin, Jenny Benois-Pineau, Renaud Péteri, Julien Morlier. Sport Action Recognition with Siamese Spatio-Temporal CNNs: Application to Table Tennis. 2018 International Conference on Content-Based Multimedia Indexing (CBMI), Sep 2018, La Rochelle, France. pp.1-6, ⟨10.1109/CBMI.2018.8516488⟩. ⟨hal-02360011⟩