Self-Attention Temporal Convolutional Network for Long-Term Daily Living Activity Detection

Rui Dai (1) , Luca Minciullo (2) , Lorenzo Garattoni (2) , Gianpiero Francesca (3) , François Bremond (1)

1 STARS - Spatio-Temporal Activity Recognition Systems
2 Toyota Motor Europe (BELGIUM)
3 Toyota Motor Europe

In this paper, we address the detection of daily living activities in long-term untrimmed videos. The detection of daily living activities is challenging due to their long temporal components, low inter-class variation and high intra-class variation. To tackle these challenges, recent approaches based on Temporal Convolutional Networks (TCNs) have been proposed. Such methods can capture long-term temporal patterns using a hierarchy of temporal convolutional filters, pooling and up sampling steps. However, as one of the important features of con-volutional networks, TCNs process a local neighborhood across time which leads to inefficiency in modeling the long-range dependencies between these temporal patterns of the video. In this paper, we propose Self-Attention-Temporal Convolutional Network (SA-TCN), which is able to capture both complex activity patterns and their dependencies within long-term untrimmed videos. We evaluate our proposed model on DAily Home LIfe Activity Dataset (DAHLIA) and Breakfast datasets. Our proposed method achieves state-of-the-art performance on both DAHLIA and Breakfast dataset.

Domaines: Apprentissage [cs.LG]; Informatique [cs]; Vision par ordinateur et reconnaissance de formes [cs.CV]

Rui Dai, Luca Minciullo, Lorenzo Garattoni, Gianpiero Francesca, François Bremond. Self-Attention Temporal Convolutional Network for Long-Term Daily Living Activity Detection. AVSS 2019 - 16th IEEE International Conference on Advanced Video and Signal-Based Surveillance (AVSS), Sep 2019, Taipei, Taiwan. ⟨hal-02357161⟩