Multi-Task Deep Learning for Pedestrian Detection, Action Recognition and Time to Cross Prediction

Danut Ovidiu Pop 1, 2 Alexandrina Rogozan 2 Clément Chatelain 3, 4 Fawzi Nashashibi 1 Abdelaziz Bensrhair 3, 2
2 STI - LITIS - Equipe Systèmes de Transport Intelligent
LITIS - Laboratoire d'Informatique, de Traitement de l'Information et des Systèmes
4 DocApp - LITIS - Equipe Apprentissage
LITIS - Laboratoire d'Informatique, de Traitement de l'Information et des Systèmes
Abstract : A pedestrian detection system is a crucial component of advanced driver assistance systems since it contributes to road flow safety. The safety of traffic participants could be significantly improved if these systems could also predict and recognize pedestrian’s actions, or even estimate the time, for each pedestrian, to cross the street. In this paper, we focus not only on pedestrian detection and pedestrian action recognition but also on estimating if the pedestrian’s action presents a risky situation according to time to cross the street. We propose 1) a pedestrian detection and action recognition component based, on RetinaNet; 2) an estimation of the time to cross the street for multiple pedestrians using a recurrent neural network. For each pedestrian, the recurrent network estimates the pedestrian’s action intention in order to predict the time to cross the street. We based our experiments on the JAAD dataset, and show that integrating multiple pedestrian action tags for the detection part when merge with a recurrent neural network (LSTM) allows a significant performance improvement.
Document type :
Journal articles
Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-02352800
Contributor : Clément Chatelain <>
Submitted on : Thursday, November 7, 2019 - 8:45:49 AM
Last modification on : Friday, November 8, 2019 - 11:19:58 AM

Links full text

Identifiers

Citation

Danut Ovidiu Pop, Alexandrina Rogozan, Clément Chatelain, Fawzi Nashashibi, Abdelaziz Bensrhair. Multi-Task Deep Learning for Pedestrian Detection, Action Recognition and Time to Cross Prediction. IEEE Access, IEEE, 2019, 7, pp.149318-149327. ⟨10.1109/ACCESS.2019.2944792⟩. ⟨hal-02352800⟩

Share

Metrics

Record views

24