BAREM: A multimodal dataset of individuals interacting with an e-service platform

Romain Belmonte (1) , Amel Aissaoui (1) , Sofiane Mihoubi (1) , Benjamin Allaert (1) , José Mennesson (1, 2) , Ioan Marius Bilasco (1, 3) , Laurent Goncalves (4)

1 CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189
2 IMT Lille Douai - Ecole nationale supérieure Mines-Télécom Lille Douai
3 LIFL - FOX MIIRE
4 SOFTEAM

The use of e-service platforms has become essential for many applications (administrative documents, online shopping, reservations). Although these platforms have improved significantly the user experience, unexpected and stressful situations can occur. Navigation problems (latency, missing information, poor ergonomics) are not always reported to the designers. To address this problem, we propose a multimodal dataset (video, audio, and physiological data) to help implicitly quantify the impact of navigation problems on users when using an e-service platform. A scenario has been designed to generate various navigation problems which can lead to changes in user behaviour. A baseline is proposed to spot changes in user behaviour, opening the way towards automatically qualifying user experiences while using e-service platforms.

Mots clés: E-service platform; multimodal dataset; behaviour analysis; anomaly detection

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]

Romain Belmonte, Amel Aissaoui, Sofiane Mihoubi, Benjamin Allaert, José Mennesson, et al.. BAREM: A multimodal dataset of individuals interacting with an e-service platform. CBMI 2021 - Content-based Multimedia Indexing, Jun 2021, Lille / Virtual, France. ⟨hal-03263944⟩