Adaptor Grammars for the Linguist: Word Segmentation Experiments for Very Low-Resource Languages

Pierre Godard 1 Laurent Besacier 2 François Yvon 1 Martine Adda-Decker 3 Gilles Adda 1 Hélène Maynard 1 Annie Rialland 3
1 LIMSI - Laboratoire d'Informatique pour la Mécanique et les Sciences de l'Ingénieur
2 GETALP - Groupe d’Étude en Traduction Automatique/Traitement Automatisé des Langues et de la Parole
LIG - Laboratoire d'Informatique de Grenoble
3 LPP - LPP - Laboratoire de Phonétique et Phonologie - UMR 7018
Abstract : Computational Language Documentation attempts to make the most recent research in speech and language technologies available to linguists working on language preservation and documentation. In this paper, we pursue two main goals along these lines. The first is to improve upon a strong baseline for the unsupervised word discovery task on two very low-resource Bantu languages, taking advantage of the expertise of linguists on these particular languages. The second consists in exploring the Adaptor Grammar framework as a decision and prediction tool for linguists studying a new language. We experiment 162 grammar configurations for each language and show that using Adaptor Grammars for word segmentation enables us to test hypotheses about a language. Specializing a generic grammar with language specific knowledge leads to great improvements for the word discovery task, ultimately achieving a leap of about 30% token F-score from the results of a strong baseline.
Keywords : Unsupervised word segmentation adaptor grammars
Document type :
Conference papers
Liste complète des métadonnées
https://hal.archives-ouvertes.fr/hal-01910757
Contributor : Limsi Publications <>
Submitted on : Thursday, November 1, 2018 - 9:48:07 PM
Last modification on : Tuesday, April 2, 2019 - 1:48:00 AM
Document(s) archivé(s) le : Saturday, February 2, 2019 - 2:06:27 PM

File

W18-5804.pdf
Files produced by the author(s)

Identifiers

Collections

LIMSI | UNIV-PARIS-SACLAY | LIG_TDCGE_GETALP | SORBONNE-UNIVERSITE | LPP | UNIV-PARIS3 | UGA | USPC | LIG

Citation

Pierre Godard, Laurent Besacier, François Yvon, Martine Adda-Decker, Gilles Adda, et al.. Adaptor Grammars for the Linguist: Word Segmentation Experiments for Very Low-Resource Languages. Workshop on Computational Research in Phonetics, Phonology, and Morphology, Oct 2018, Bruxelles, Belgium. pp.32 - 42, ⟨10.18653/v1/P17⟩. ⟨hal-01910757⟩

Export

BibTeX TEI DC DCterms EndNote

Share

Metrics

Record views

66

Files downloads

41

Contact

support.ccsd.cnrs.fr
hal.support@ccsd.cnrs.fr
Logo CCSD