Curriculum-based transfer learning for an effective end-to-end spoken language understanding and domain portability

Antoine Caubrière (1) , Natalia Tomashenko (2) , Antoine Laurent (1) , Emmanuel Morin (3, 4) , Nathalie Camelin (1) , Yannick Estève (2)

1 LIUM - Laboratoire d'Informatique de l'Université du Mans
2 LIA - Laboratoire Informatique d'Avignon
3 LS2N - Laboratoire des Sciences du Numérique de Nantes
4 LS2N - équipe TALN - Traitement Automatique du Langage Naturel

We present an end-to-end approach to extract semantic concepts directly from the speech audio signal. To overcome the lack of data available for this spoken language understanding approach, we investigate the use of a transfer learning strategy based on the principles of curriculum learning. This approach allows us to exploit out-of-domain data that can help to prepare a fully neural architecture. Experiments are carried out on the French MEDIA and PORTMEDIA corpora and show that this end-to-end SLU approach reaches the best results ever published on this task. We compare our approach to a classical pipeline approach that uses ASR, POS tagging, lemmatizer, chunker... and other NLP tools that aim to enrich ASR outputs that feed an SLU text to concepts system. Last, we explore the promising capacity of our end-to-end SLU approach to address the problem of domain portability.

Mots clés: domain portability; curriculum learning; neural network; Index Terms: spoken language understanding; transfer learning

Domaines: Intelligence artificielle [cs.AI]; Réseau de neurones [cs.NE]; Apprentissage [cs.LG]; Autre [cs.OH]

Antoine Caubrière, Natalia Tomashenko, Antoine Laurent, Emmanuel Morin, Nathalie Camelin, et al.. Curriculum-based transfer learning for an effective end-to-end spoken language understanding and domain portability. 20th Annual Conference of the International Speech Communication Association (InterSpeech), Sep 2019, Graz, Austria. pp.1198-1202, ⟨10.21437/interspeech.2019-1832⟩. ⟨hal-02304597⟩