Experimenting the Automatic Recognition of Non-Conventionalized Units in Sign Language - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Algorithms Année : 2020

Experimenting the Automatic Recognition of Non-Conventionalized Units in Sign Language

Résumé

Sign Languages are visual-gestural languages that have developed naturally in Deaf communities. They are based on the use of lexical signs, that is conventionalized units, as well as highly iconic structures, i.e. when the form of an utterance and the meaning it carries are not independent. Although most research in automatic Sign Language Recognition has focused on lexical signs, we wish to broaden this perspective and consider the recognition of non-conventionalized iconic and syntactic elements. We propose to use corpora made by linguists like the finely and consistently annotated dialogue corpus Dicta-Sign-LSF-v2. We then redefine the problem of automatic Sign Language Recognition as the recognition of linguistic descriptors, with carefully thought out performance metrics. Moreover, we develop a compact and generalizable representation of signers in videos by parallel processing of the hands, face and upper body, then an adapted learning architecture based on a Recurrent Convolutional Neural Network. Through a study focused on the recognition of four linguistic descriptors, we show the soundness of the proposed approach and pave the way for a wider understanding of Continuous Sign Language Recognition.
Fichier principal
Vignette du fichier
algo.pdf (5.4 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03060271 , version 1 (16-12-2020)

Identifiants

Citer

Valentin Belissen, Annelies Braffort, Michèle Gouiffès. Experimenting the Automatic Recognition of Non-Conventionalized Units in Sign Language. Algorithms, 2020, 13, pp.310-345. ⟨10.3390/a13120310⟩. ⟨hal-03060271⟩
69 Consultations
59 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More