Dicta-Sign-LSF-v2: Remake of a Continuous French Sign Language Dialogue Corpus and a First Baseline for Automatic Sign Language Processing

Dicta-Sign-LSF-v2: Refonte d'un corpus de dialogue continu en Langue des Signes Française et une première référence en traitement automatique de langue des signes

Valentin Belissen (1, 2, 3) , Annelies Braffort (3) , Michèle Gouiffès (3, 4, 5)

1 Université Paris-Saclay
2 UP11 - Université Paris-Sud - Paris 11
3 LIMSI - Laboratoire d'Informatique pour la Mécanique et les Sciences de l'Ingénieur
4 LISN - Laboratoire Interdisciplinaire des Sciences du Numérique
5 AMI - Architectures et Modèles pour l'Interaction - LISN

While the research in automatic Sign Language Processing (SLP) is growing, it has been almost exclusively focused on recognizing lexical signs, whether isolated or within continuous SL production. However, Sign Languages include many other gestural units like iconic structures, which need to be recognized in order to go towards a true SL understanding. In this paper, we propose a newer version of the publicly available SL corpus Dicta-Sign, limited to its French Sign Language part. Involving 16 different signers, this dialogue corpus was produced with very few constraints on the style and content. It includes lexical and non-lexical annotations over 11 hours of video recording, with 35000 manual units. With the aim of stimulating research in SL understanding, we also provide a baseline for the recognition of lexical signs and non-lexical structures on this corpus. A very compact modeling of a signer is built and a Convolutional-Recurrent Neural Network is trained and tested on Dicta-Sign-LSF-v2, with state-of-the-art results, including the ability to detect iconicity in SL production.

Mots clés: Sign Language Recognition; Sign Language Processing; Iconicity; Depicting Signs; French Sign Language

Domaines: Traitement du signal et de l'image [eess.SP]; Vision par ordinateur et reconnaissance de formes [cs.CV]; Linguistique; Machine Learning [stat.ML]; Apprentissage [cs.LG]; Intelligence artificielle [cs.AI]; Traitement des images [eess.IV]; Informatique [cs]

Valentin Belissen, Annelies Braffort, Michèle Gouiffès. Dicta-Sign-LSF-v2: Remake of a Continuous French Sign Language Dialogue Corpus and a First Baseline for Automatic Sign Language Processing. LREC 2020, 12th Conference on Language Resources and Evaluation, 2020, Marseille, France. ⟨hal-02541792⟩