Sign Language Tutoring Tool

Oya Aran (1) , Ismail Ari (1) , Alexandre Benoit (2) , Ana Huerta Carrillo (3) , François-Xavier Fanard (4) , Pavel Campr (5) , Lale Akarun (1) , Alice Caplier (2) , Michèle Rombaut (2) , Bulent Sankur (1)

1 Boǧaziçi üniversitesi = Boğaziçi University [Istanbul]
2 GIPSA-GPIG - GIPSA - Géométrie, Perception, Images, Geste
3 UPM - Universidad Politécnica de Madrid
4 TELE - Laboratoire de Télécommunications et Télédétection [Louvain]
5 University of West Bohemia [Plzeň ]

In this project, we have developed a sign language tutor that lets users learn isolated signs by watching recorded videos and by trying the same signs. The system records the user's video and analyses it. If the sign is recognized, both verbal and animated feedback is given to the user. The system is able to recognize complex signs that involve both hand gestures and head movements and expressions. Our performance tests yield a 99% recognition rate on signs involving only manual gestures and 85% recognition rate on signs that involve both manual and non manual components, such as head movement and facial expressions.

Mots clés: Gesture recognition; sign language recognition; head movement analysis; human body animation

Domaines: Traitement du signal et de l'image [eess.SP]; Apprentissage [cs.LG]; Informatique ubiquitaire; Interface homme-machine [cs.HC]; Traitement du signal et de l'image [eess.SP]

Oya Aran, Ismail Ari, Alexandre Benoit, Ana Huerta Carrillo, François-Xavier Fanard, et al.. Sign Language Tutoring Tool. eNTERFACE'06. Summer Workshop. on Multimodal Interfaces, 2007, Dubrovnik, Croatia. ⟨hal-00256661⟩