Unsupervised Word Discovery Using Attentional Encoder-Decoder Models

Marcely Zanon Boito (1, 2, 3) , Laurent Besacier (1) , Aline Villavicencio (2)

1 LIG - Laboratoire d'Informatique de Grenoble
2 UFRGS - Instituto de Informática da UFRGS
3 UGA UFR IMAG - Université Grenoble Alpes - UFR Informatique et Mathématiques Appliquées

Attention-based sequence-to-sequence neural machine translation systems have been shown to jointly align and translate source sentences into target sentences. In this project we use unsegmented symbol sequences (characters and phonemes) as source, aiming to explore the soft-alignment probability matrices generated during training and to evaluate if these soft-alignments allow us to discover latent lexicon representations. If successful, such approach could be useful for documenting unwritten and/or endangered languages. However, for this to be feasible, attention models should be robust to low-resource scenarios, of several thousand of sentences only. We use a parallel corpus between the endangered language Mboshi and French, as well as a larger and more controlled English-French parallel corpus. Our goal is to explore different representation levels and study their impact, together with the impact of different data set sizes, in the quality of the generated soft-alignment probability matrices.

Domaines: Informatique et langage [cs.CL]; Informatique [cs]

Marcely Zanon Boito, Laurent Besacier, Aline Villavicencio. Unsupervised Word Discovery Using Attentional Encoder-Decoder Models. WiNLP workshop, ACL 2017, Jul 2017, Vancouver, Canada. ⟨hal-02895851⟩