Skip to Main content Skip to Navigation
Journal articles

Graphical models for social behavior modeling in face-to face interaction

Alaeddine Mihoub 1, 2 Gérard Bailly 2 Christian Wolf 1 Frédéric Elisei 3, 2
1 imagine - Extraction de Caractéristiques et Identification
LIRIS - Laboratoire d'InfoRmatique en Image et Systèmes d'information
3 GIPSA-Services - GIPSA-Services
GIPSA-lab - Grenoble Images Parole Signal Automatique
Abstract : The goal of this paper is to model the coverbal behavior of a subject involved in face-to-face social interactions. For this end, we present a multimodal behavioral model based on a Dynamic Bayesian Network (DBN). The model was inferred from multimodal data of interacting dyads in a specific scenario designed to foster mutual attention and multimodal deixis of objects and places in a collaborative task. The challenge for this behavioral model is to generate coverbal actions (gaze, hand gestures) for the subject given his verbal productions, the current phase of the interaction and the perceived actions of the partner. In our work, the structure of the DBN was learned from data, which revealed an interesting causality graph describing precisely how verbal and coverbal human behaviors are coordinated during the studied interactions. Using this structure, DBN exhibits better performances compared to classical baseline models such as Hidden Markov Models (HMMs) and Hidden Semi-Markov Models (HSMMs). We outperform the baseline in both measures of performance, i.e. interaction unit recognition and behavior generation. DBN also reproduces more faithfully the coordination patterns between modalities observed in ground truth compared to the baseline models.
Complete list of metadata

Cited literature [36 references]  Display  Hide  Download
Contributor : Christian Wolf <>
Submitted on : Friday, October 21, 2016 - 6:28:28 PM
Last modification on : Tuesday, June 1, 2021 - 2:08:09 PM
Long-term archiving on: : Friday, February 3, 2017 - 12:24:12 PM


Files produced by the author(s)



Alaeddine Mihoub, Gérard Bailly, Christian Wolf, Frédéric Elisei. Graphical models for social behavior modeling in face-to face interaction. Pattern Recognition Letters, Elsevier, 2016, 74, pp.82-89. ⟨10.1016/j.patrec.2016.02.005⟩. ⟨hal-01279427⟩



Record views


Files downloads