Robust Spatiotemporal Convolutional Long Short-Term Memory Algorithm for Video Prediction

Wael Saideni (1, 2) , Fabien Courrèges (1, 2) , David Helbert (3, 4) , Jean Pierre P Cances (1, 2)

1 XLIM-SRI - Systèmes et Réseaux Intelligents
2 UNILIM - Université de Limoges
3 Université de Poitiers - Faculté de Sciences fondamentales et appliquées
4 XLIM-ASALI - Synthèse et analyse d'images

The use of recurrent neural networks in several applications has allowed to capture impressive results, especially in various applications such as video prediction and it has become a promising direction of scientific research. In this paper, we introduce a novel algorithm for video prediction called "Robust Spatiotemporal Convolutional Long Short-Term Memory (Robust-ST-ConvLSTM) Algorithm" that outperforms the state-of-the-art approaches. Robust-ST-ConvLSTM is a memory flow algorithm based on higher order ConvLSTM. This memory flow algorithm is holding the spatiotemporal information to optimize and control the prediction abilities of the ConvLSTM cell. Our approach is developed in the specific context of predicting future frames based on historical observations, and we experimentally validate the ability of the proposed algorithm on two spatiotemporal datasets, including a moving variant of MNIST dataset of handwritten digits, and KTH which is a human motion dataset.

Mots clés: Video Prediction; Deep Learning; Neural Networks; Computer Vision; ConvLSTM; Memory Flow; Hidden States

Domaines: Traitement du signal et de l'image [eess.SP]

Wael Saideni, Fabien Courrèges, David Helbert, Jean Pierre P Cances. Robust Spatiotemporal Convolutional Long Short-Term Memory Algorithm for Video Prediction. 2021. ⟨hal-03836816⟩