Mining a Multimodal Corpus of Doctor's Training for Virtual Patient's Feedbacks

Chris Porhet (1) , Magalie Ochs (2, 3, 4) , Jorane Saubesty (1) , Grégoire De Montcheuil (1) , Roxane Bertrand (1)

1 LPL - Laboratoire Parole et Langage
2 DIMAG - Data, Information & content MAnagement Group
3 LIS - Laboratoire d'Informatique et des Systèmes (LIS) (Marseille, Toulon)
4 R2I - Recherche d’information et Interactions

Doctors should be trained not only to perform medical or surgical acts but also to develop competences in communication for their interaction with patients. For instance, the way doctors deliver bad news has a signifcant impact on the therapeutic process. In order to facilitate the doctors' training to break bad news, we aim at developing a virtual patient ables to interact in a multimodal way with doctors announcing an undesirable event. One of the key elements to create an engaging interaction is the feedbacks' behavior of the virtual character. In order to model the virtual patient's feedbacks in the context of breaking bad news, we have analyzed a corpus of real doctor's training. Te verbal and nonverbal signals of both the doctors and the patients have been annotated. In order to identify the types of feedbacks and the elements that may elicit a feedback, we have explored the corpus based on sequences mining methods. Rules, that have been extracted from the corpus, enable us to determine when a virtual patient should express which feedbacks when a doctor announces a bad new.

Mots clés: Virtual reality; multimodal corpora; Embodied conversational agent; rules mining; feedbacks; health domain

Domaines: Linguistique; Système multi-agents [cs.MA]; Intelligence artificielle [cs.AI]

Chris Porhet, Magalie Ochs, Jorane Saubesty, Grégoire De Montcheuil, Roxane Bertrand. Mining a Multimodal Corpus of Doctor's Training for Virtual Patient's Feedbacks. 19th International Conference on Multimodal Interaction (ICMI), Nov 2017, Glasgow, United Kingdom. ⟨10.1145/3136755.3136816⟩. ⟨hal-01654812⟩