Skip to Main content Skip to Navigation
Conference papers

Using Monolingual Data in Neural Machine Translation: a Systematic Study

Franck Burlot 1, 2 François Yvon 2
2 TLP - Traitement du Langage Parlé
LIMSI - Laboratoire d'Informatique pour la Mécanique et les Sciences de l'Ingénieur : 247329
Abstract : Neural Machine Translation (MT) has radically changed the way systems are developed. A major difference with the previous generation (Phrase-Based MT) is the way monolingual target data, which often abounds, is used in these two paradigms. While Phrase-Based MT can seamlessly integrate very large language models trained on billions of sentences, the best option for Neural MT developers seems to be the generation of artificial parallel data through \textsl{back-translation} - a technique that fails to fully take advantage of existing datasets. In this paper, we conduct a systematic study of back-translation, comparing alternative uses of monolingual data, as well as multiple data generation procedures. Our findings confirm that back-translation is very effective and give new explanations as to why this is the case. We also introduce new data simulation techniques that are almost as effective, yet much cheaper to implement.
Complete list of metadatas

Cited literature [36 references]  Display  Hide  Download
Contributor : Limsi Publications <>
Submitted on : Sunday, December 2, 2018 - 8:38:50 PM
Last modification on : Monday, February 10, 2020 - 6:14:12 PM
Document(s) archivé(s) le : Sunday, March 3, 2019 - 12:23:12 PM


Publisher files allowed on an open archive


  • HAL Id : hal-01910235, version 1


Franck Burlot, François Yvon. Using Monolingual Data in Neural Machine Translation: a Systematic Study. Conference on Machine Translation, Oct 2018, Brussels, Belgium. ⟨hal-01910235⟩



Record views


Files downloads