The SignCom System for Data-Driven Animation of Interactive Virtual Signers : Methodology and Evaluation

Sylvie Gibet (1) , Nicolas Courty (1) , Kyle Duarte (1) , Thibaut Le Naour (1)

1 VALORIA - Laboratoire de Recherche en Informatique et ses Applications de Vannes et Lorient

In this paper we present a multichannel animation system for producing utterances signed in French Sign Language (LSF) by a virtual character. The main challenges of such a system are simultaneously capturing data for the entire body, including the movements of the torso, hands, and face, and developing a data-driven animation engine that takes into account the expressive characteristics of signed languages. Our approach consists of decomposing motion along different channels, representing the body parts that correspond to the linguistic components of signed languages. We show the ability of this animation system to create novel utterances in LSF, and present an evaluation by target users which highlights the importance of the respective body parts in the production of signs. We validate our framework by testing the believability and intelligibility of our virtual signer.

Mots clés: Communicative gestures; virtual signer; sign language; data-driven animation; multimodal animation; evaluation

Domaines: Interface homme-machine [cs.HC]; Traitement du signal et de l'image [eess.SP]; Traitement du signal et de l'image [eess.SP]

Sylvie Gibet, Nicolas Courty, Kyle Duarte, Thibaut Le Naour. The SignCom System for Data-Driven Animation of Interactive Virtual Signers : Methodology and Evaluation. ACM Transactions on Interactive Intelligent Systems , 2011, 1 (1), pp.6. ⟨hal-00664705⟩