Social behavior modeling based on Incremental Discrete Hidden Markov Models

Alaeddine Mihoub (1, 2) , Gérard Bailly (2) , Christian Wolf (1)

1 imagine - Extraction de Caractéristiques et Identification
2 GIPSA-MAGIC - GIPSA - Machines parlantes, Gestes oro-faciaux, Interaction Face-à-face, Communication augmentée

Modeling multimodal face-to-face interaction is a crucial step in the process of building social robots or users-aware Embodied Conversational Agents (ECA). In this context, we present a novel approach for human behavior analysis and generation based on what we called "Incremental Discrete Hidden Markov Model" (IDHMM). Joint multimodal activities of interlocutors are first modeled by a set of DHMMs that are specific to supposed joint cognitive states of the interlocutors. Respecting a task-specific syntax, the IDHMM is then built from these DHMMs and split into i) a recognition model that will determine the most likely sequence of cognitive states given the multimodal activity of the in- terlocutor, and ii) a generative model that will compute the most likely activity of the speaker given this estimated sequence of cognitive states. Short-Term Viterbi (STV) decoding is used to incrementally recognize and generate behav- ior. The proposed model is applied to parallel speech and gaze data of interact- ing dyads.

Mots clés: Face to face interaction; behavior model; action-perception loops; cognitive state recognition; gaze generation; HMM; Online Viterbi decoding; latency

Domaines: Traitement du signal et de l'image [eess.SP]; Traitement du signal et de l'image [eess.SP]

Alaeddine Mihoub, Gérard Bailly, Christian Wolf. Social behavior modeling based on Incremental Discrete Hidden Markov Models. Human Behavior Understanding. 4th International Workshop, HBU 2013, Barcelona, Spain, October 22, 2013. Proceedings, Springer International Publishing, pp.172-183, 2013, Lecture Notes in Computer Science, n°8212, 978-3-319-02714-2. ⟨10.1007/978-3-319-02714-2_15⟩. ⟨hal-00851903⟩