A Multimodal Model for Predicting Conversational Feedbacks

Auriane Boudin (1) , Roxane Bertrand (1) , Stéphane Rauzy (1) , Magalie Ochs (2, 3) , Philippe Blache (1)

1 LPL - Laboratoire Parole et Langage
2 ILCB - Institute of Language, Communication and the Brain
3 TALEP - Traitement Automatique du Langage Ecrit et Parlé

We propose in this paper a statistical model in the perspective of predicting listener's feedbacks in a conversation. The first contribution of the paper is a study of the prediction of all feedbacks, including those in overlap with the speaker with a good accuracy. Existing model are good at predicting feedbacks during a pause, but reach a very low success level for all feedbacks. We give in this paper a first step towards this complex problem. The second contribution is a model predicting precisely the type of the feedback (generic vs. specific) as well as other specific features (valence expectation) useful in particular for generating feedbacks in dialogue systems. This work relies on an original corpus.

Mots clés: Feedback; Linguistic interaction; Statistical model; Corpus study

Domaines: Intelligence artificielle [cs.AI]; Interface homme-machine [cs.HC]; Traitement du texte et du document; Linguistique; Sciences cognitives

Auriane Boudin, Roxane Bertrand, Stéphane Rauzy, Magalie Ochs, Philippe Blache. A Multimodal Model for Predicting Conversational Feedbacks. International Conference on Text, Speech, and Dialogue (TSD ), 2021, Olomouc, Czech Republic. ⟨10.1007/978-3-030-83527-9_46⟩. ⟨hal-03331446v2⟩