Graphical models for social behavior modeling in face-to face interaction

Alaeddine Mihoub 1, 2 Gérard Bailly 2 Christian Wolf 1 Frédéric Elisei 3, 2
1 imagine - Extraction de Caractéristiques et Identification
LIRIS - Laboratoire d'InfoRmatique en Image et Systèmes d'information
2 GIPSA-CRISSP - CRISSP
GIPSA-DPC - Département Parole et Cognition
3 GIPSA-Services - GIPSA-Services
GIPSA-lab - Grenoble Images Parole Signal Automatique
Abstract : The goal of this paper is to model the coverbal behavior of a subject involved in face-to-face social interactions. For this end, we present a multimodal behavioral model based on a Dynamic Bayesian Network (DBN). The model was inferred from multimodal data of interacting dyads in a specific scenario designed to foster mutual attention and multimodal deixis of objects and places in a collaborative task. The challenge for this behavioral model is to generate coverbal actions (gaze, hand gestures) for the subject given his verbal productions, the current phase of the interaction and the perceived actions of the partner. In our work, the structure of the DBN was learned from data, which revealed an interesting causality graph describing precisely how verbal and coverbal human behaviors are coordinated during the studied interactions. Using this structure, DBN exhibits better performances compared to classical baseline models such as Hidden Markov Models (HMMs) and Hidden Semi-Markov Models (HSMMs). We outperform the baseline in both measures of performance, i.e. interaction unit recognition and behavior generation. DBN also reproduces more faithfully the coordination patterns between modalities observed in ground truth compared to the baseline models.
Complete list of metadatas

Cited literature [36 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01279427
Contributor : Christian Wolf <>
Submitted on : Friday, October 21, 2016 - 6:28:28 PM
Last modification on : Friday, June 14, 2019 - 2:54:37 PM
Long-term archiving on : Friday, February 3, 2017 - 12:24:12 PM

File

PRL_R2.pdf
Files produced by the author(s)

Identifiers

Citation

Alaeddine Mihoub, Gérard Bailly, Christian Wolf, Frédéric Elisei. Graphical models for social behavior modeling in face-to face interaction. Pattern Recognition Letters, Elsevier, 2016, 74, pp.82-89. ⟨10.1016/j.patrec.2016.02.005⟩. ⟨hal-01279427⟩

Share

Metrics

Record views

711

Files downloads

259