The Badalona Corpus An Audio, Video and Neuro-Physiological Conversational Dataset

Philippe Blache (1) , Salomé Antoine , Dorina de Jong , Lena-Marie Huttner , Emilia Kerr , Thierry Legou , Eliot Maës , Clément François

1 LPL - Laboratoire Parole et Langage

We present in this paper the first natural conversational corpus recorded with all modalities and neuro-physiological signals. Five dyads (10 participants, Spanish native speakers) have been recorded three times, during three sessions (about 30 minutes each) with 4 days interval. During each session, audio and video are captured as well as the neural signal (EEG with Emotiv-EPOC) and the electro-physiological one (with Empatica-E4). This resource is original in several respects. Technically, it is the first one gathering all these types of data in a natural conversation situation. Moreover, the recording of the same dyads at different periods opens the door to new longitudinal investigations such as the evolution of interlocutors' alignment over time. The paper situates this new type of resources in the literature, presents the experimental setup and describes different annotations enriching the corpus.

Mots clés: Multimodal corpus; natural conversation; convergence; neuro-physiological data; EEG

Domaines: Intelligence artificielle [cs.AI]; Interface homme-machine [cs.HC]; Traitement du texte et du document; Linguistique; Sciences cognitives

Philippe Blache, Salomé Antoine, Dorina de Jong, Lena-Marie Huttner, Emilia Kerr, et al.. The Badalona Corpus An Audio, Video and Neuro-Physiological Conversational Dataset. Language Resources and Evaluation Conference, 2022, Marseille, France. ⟨hal-03657477⟩