Reference interpretation in a multimodal environment combining speech and gesture

Nadia Bellalem 1 Laurent Romary 1
1 DIALOGUE - DIALOGUE
CRIN - Centre de Recherche en Informatique de Nancy, UHP - Université Henri Poincaré - Nancy 1, Université Nancy 2, INPL - Institut National Polytechnique de Lorraine
Abstract : The study presented in this paper is dedicated to the integration of pointing gestures within a task oriented man-machine dialogue system. From our point of view, it is natural to see this problem as a sub-part of the more general study of reference in a dialogue since we took the option to limit the analysis of gestures to those explicitly associated with speech and more specifically for singling out objects in the task. Even under these hypotheses, understanding the global message resulting from the combinaison of speech and gesture implies that a precise analysis of the gestural trajectory should be done. This analysis may be split up into two main steps. The first, which we may call structural, is centred upon the shape of the gestural trajectory to mark the meaningful parts to which a specific designation role will be given. The second step has to do with the contextual interpretation of the gesture for which one has to take into account a) the features of the application and the way it is visualized and b) the oral dialogue and more specifically the instructional content of the referring expression accompanying the gesture.
Type de document :
Communication dans un congrès
First International Workshop on Intelligence and Multimodality in Multimedia Interfaces, 1995, Edinburgh, United Kingdom. 5 p., 1995
Liste complète des métadonnées

Littérature citée [8 références]  Voir  Masquer  Télécharger

https://hal.archives-ouvertes.fr/hal-00521585
Contributeur : Laurent Romary <>
Soumis le : mardi 5 août 2014 - 15:22:44
Dernière modification le : jeudi 11 janvier 2018 - 06:23:18
Document(s) archivé(s) le : lundi 17 novembre 2014 - 15:41:55

Fichier

ReferenceInterpretation.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-00521585, version 1

Collections

Citation

Nadia Bellalem, Laurent Romary. Reference interpretation in a multimodal environment combining speech and gesture. First International Workshop on Intelligence and Multimodality in Multimedia Interfaces, 1995, Edinburgh, United Kingdom. 5 p., 1995. 〈hal-00521585〉

Partager

Métriques

Consultations de la notice

247

Téléchargements de fichiers

66