Skip to Main content Skip to Navigation
Conference papers

Analyse multimodale d'interaction humaine dans le cockpit d'un véhicule

Quentin Portes 1, 2, 3 Julien Pinquier 2 Frédéric Lerasle 3 Jose Mendes-Carlalho 1 
3 LAAS-RAP - Équipe Robotique, Action et Perception
LAAS - Laboratoire d'analyse et d'architecture des systèmes
Abstract : Nowadays, every car maker is thinking about the future of mobility. Electric vehicles, autonomous vehicles and sharing vehicles are the most promising opportunities. The lack of control authority in autonomous and sharing vehicles raises different issues like the passenger safety. To ensure it, new systems able to understand interactions and possible conflicts between passengers have to be designed. They should be able to predict and trigger with high accuracy, an alert to a remote controller before a critical situation happens in the cockpit. In order to better understand the features of these insecure situations, we recorded an audio-video dataset in real vehicle context. Twenty-two participants playing three different scenarios ("curious","argued refusal" and "not argued refusal") of interactionsbetween a driver and a passenger were recorded. We propose a deep learning approach which achieves a balanced accuracy of 81%. Practically, we highlight that combining multimodality, namely video, audio and text as well as temporality are the keys to perform such accurate predictions in scenario recognition.
Complete list of metadata
Contributor : CCSD Connect in order to contact the contributor
Submitted on : Thursday, September 9, 2021 - 3:39:21 PM
Last modification on : Monday, July 4, 2022 - 9:45:41 AM
Long-term archiving on: : Saturday, December 11, 2021 - 7:25:15 AM


Files produced by the author(s)


  • HAL Id : hal-03339623, version 1


Quentin Portes, Julien Pinquier, Frédéric Lerasle, Jose Mendes-Carlalho. Analyse multimodale d'interaction humaine dans le cockpit d'un véhicule. 18èmes journées francophones des jeunes chercheurs en vision par ordinateur (ORASIS 2021), Centre National de la Recherche Scientifique [CNRS]; Equipe REVA, IRIT : Institut de Recherche en Informatique de Toulouse, Sep 2021, Saint Ferréol, France. ⟨hal-03339623⟩



Record views


Files downloads