Detecting small group activities from multimodal observations

Oliver Brdiczka (1) , Jérôme Maisonnasse (1) , Patrick Reignier (1) , James L. Crowley (1)

1 PRIMA - Perception, recognition and integration for observation of activity

This article addresses the problem of detecting configurations and activities of small groups of people in an augmented environment. The proposed approach takes a continuous stream of observations coming from differ- ent sensors in the environment as input. The goal is to separate distinct distributions of these observations corre- sponding to distinct group configurations and activities. This article describes an unsupervised method based on the cal- culation of the Jeffrey divergence between histograms over observations. These histograms are generated from adjacent windows of variable size slid from the beginning to the end of a meeting recording. The peaks of the resulting Jeffrey di- vergence curves are detected using successive robust mean estimation. After a merging and filtering process, the re- tained peaks are used to select the best model, i.e. the best allocation of observation distributions for a meeting record- ing. These distinct distributions can be interpreted as distinct segments of group configuration and activity. To evaluate this approach, 5 small group meetings, one seminar and one cocktail party meeting have been recorded. The observations A short version of this article [6] obtained the Best Paper Award of the 3rd IFIP Conference on Artificial Intelligence Applications and Innovations (AIAI) 2006. O. Brdiczka (ﰌ) * J. Maisonnasse * P. Reignier * J.L. Crowley INRIA Rhône-Alpes, 655 avenue de l'Europe, 38334 Saint Ismier Cedex, France e-mail: [e-mail] J. Maisonnasse e-mail: [e-mail] P. Reignier e-mail: [e-mail] J.L. Crowley e-mail: [e-mail] of the small groups meetings and the seminar were gener- ated by a speech activity detector, while the observations of the cocktail party meeting were generated by both the speech activity detector and a visual tracking system. The authors measured the correspondence between detected seg- ments and labeled group configurations and activities. The obtained results are promising, in particular as the method is completely unsupervised.

Mots clés: Attention; Multi-modal interaction

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]; Informatique ubiquitaire

Oliver Brdiczka, Jérôme Maisonnasse, Patrick Reignier, James L. Crowley. Detecting small group activities from multimodal observations. Applied Intelligence, 2009, 30 (1), pp.47-56. ⟨hal-00665329⟩