Learning joint multimodal behaviors for face-to-face interaction: performance & properties of statistical models

Gérard Bailly (1) , Alaeddine Mihoub (2, 1) , Christian Wolf (2) , Frédéric Elisei (3, 1)

1 GIPSA-CRISSP - GIPSA - Cognitive Robotics, Interactive Systems, & Speech Processing
2 imagine - Extraction de Caractéristiques et Identification
3 GIPSA-Services - GIPSA-Services

We evaluate here the ability of statistical models, namely Hidden Markov Models (HMMs) and Dynamic Bayesian Networks (DBNs), in capturing the interplay and coordination between multimodal behaviors of two individuals involved in a face-to-face interaction. We structure the intricate sensory-motor coupling of the joint multimodal scores by segmenting the whole interaction into so-called interaction units (IU). We show that the proposed statistical models are able to capture the natural dynamics of the interaction and that DBNs are particularly suitable for reproducing original distributions of so-called coordination histograms.

Domaines: Sciences de l'ingénieur [physics]

Gérard Bailly, Alaeddine Mihoub, Christian Wolf, Frédéric Elisei. Learning joint multimodal behaviors for face-to-face interaction: performance & properties of statistical models. Human-Robot Interaction. Workshop on Behavior Coordination between Animals, Humans, and Robots, Mar 2015, Portland, United States. ⟨hal-01110290⟩