Empirical Evaluation of Sequence-to-Sequence Models for Word Discovery in Low-resource Settings

Abstract : Since Bahdanau et al. [1] first introduced attention for neural machine translation, most sequence-to-sequence models made use of attention mechanisms [2, 3, 4]. While they produce soft-alignment matrices that could be interpreted as alignment between target and source languages, we lack metrics to quantify their quality, being unclear which approach produces the best alignments. This paper presents an empirical evaluation of 3 of the main sequence-to-sequence models for word discovery from unsegmented phoneme sequences: CNN, RNN and Transformer-based. This task consists in aligning word sequences in a source language with phoneme sequences in a target language, inferring from it word segmentation on the target side [5]. Evaluating word segmentation quality can be seen as an extrinsic evaluation of the soft-alignment matrices produced during training. Our experiments in a low-resource scenario on Mboshi and English languages (both aligned to French) show that RNNs surprisingly outperform CNNs and Transformer for this task. Our results are confirmed by an intrinsic evaluation of alignment quality through the use Average Normalized Entropy (ANE). Lastly, we improve our best word discovery model by using an alignment entropy confidence measure that accumulates ANE over all the occurrences of a given alignment pair in the collection.
Document type :
Conference papers
Complete list of metadatas

Cited literature [27 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-02193867
Contributor : Laurent Besacier <>
Submitted on : Wednesday, July 24, 2019 - 6:45:23 PM
Last modification on : Friday, July 26, 2019 - 1:29:01 AM

File

IS2019marcely-camera-ready.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02193867, version 1

Collections

Citation

Marcely Zanon Boito, Aline Villavicencio, Laurent Besacier. Empirical Evaluation of Sequence-to-Sequence Models for Word Discovery in Low-resource Settings. Interspeech 2019, Sep 2019, Graz, Austria. ⟨hal-02193867⟩

Share

Metrics

Record views

16

Files downloads

11