3D attention mechanism for fine-grained classification of table tennis strokes using a Twin Spatio-Temporal Convolutional Neural Networks

Pierre-Etienne Martin (1, 2) , Jenny Benois-Pineau (1) , Renaud Péteri , Julien Morlier

1 LaBRI - Laboratoire Bordelais de Recherche en Informatique
2 UB - Université de Bordeaux

The paper addresses the problem of recognition of actions in video with low inter-class variability such as Table Tennis strokes. Two stream, "twin" convolutional neural networks are used with 3D convolutions both on RGB data and optical flow. Actions are recognized by classification of temporal windows. We introduce 3D attention modules and examine their impact on classification efficiency. In the context of the study of sportsmen performances, a corpus of the particular actions of table tennis strokes is considered. The use of attention blocks in the network speeds up the training step and improves the classification scores up to 5% with our twin model. We visualize the impact on the obtained features and notice correlation between attention and player movements and position. Score comparison of state-of-the-art action classification method and proposed approach with attentional blocks is performed on the corpus. Proposed model with attention blocks outperforms previous model without them and our baseline.

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]; Traitement des images [eess.IV]; Intelligence artificielle [cs.AI]; Traitement du signal et de l'image [eess.SP]; Réseau de neurones [cs.NE]; Multimédia [cs.MM]; Apprentissage [cs.LG]; Interface homme-machine [cs.HC]

Pierre-Etienne Martin, Jenny Benois-Pineau, Renaud Péteri, Julien Morlier. 3D attention mechanism for fine-grained classification of table tennis strokes using a Twin Spatio-Temporal Convolutional Neural Networks. 25th International Conference on Pattern Recognition (ICPR2020), Jan 2021, Milano, Italy. ⟨hal-02977646⟩