Gestural Interfaces for Hearing-Impaired Communication

Oya Aran (1) , Thomas Burger (2) , Lale Akarun (1) , Alice Caplier (2)

1 Department of Computer Engineering [Bogazici]
2 GIPSA-GPIG - GIPSA - Géométrie, Perception, Images, Geste

Gestural interfaces, besides providing natural means of human computer interaction for everyone, enable the hearing impaired to use sign language or better understand speech through vision. This chapter overviews (1) the various modalities involved in gestured languages (2) the mean to automatically apprehend them individually and (3) to fuse them in order to provide a communication medium adapted to hearing-impaired. We present two example applications, a sign language tutoring tool and a cued speech interpreter and discuss theoretical and practical aspects.

Domaines: Vision par ordinateur et reconnaissance de formes [cs.CV]; Interface homme-machine [cs.HC]; Traitement des images [eess.IV]; Apprentissage [cs.LG]

Oya Aran, Thomas Burger, Lale Akarun, Alice Caplier. Gestural Interfaces for Hearing-Impaired Communication. Dimitros Tzovaras. Multimodal user interfaces: from signals to interaction, Springer, pp.219-250, 2008, Signals and Communication Technology, ⟨10.1007/978-3-540-78345-9⟩. ⟨hal-00327419⟩