Better Evaluation of ASR in Speech Translation Context Using Word Embeddings

Abstract : This paper investigates the evaluation of ASR in spoken language translation context. More precisely, we propose a simple extension of WER metric in order to penalize differently substitution errors according to their context using word embeddings. For instance, the proposed metric should catch near matches (mainly morphological variants) and penalize less this kind of error which has a more limited impact on translation performance. Our experiments show that the correlation of the new proposed metric with SLT performance is better than the one of WER. Oracle experiments are also conducted and show the ability of our metric to find better hypotheses (to be translated) in the ASR N-best. Finally, a preliminary experiment where ASR tuning is based on our new metric shows encouraging results. For reproductible experiments, the code allowing to call our modified WER and the corpora used are made available to the research community.
Type de document :
Communication dans un congrès
Interspeech 2016, Sep 2016, San-Francisco, United States. Interspeech 2016 proceedings
Liste complète des métadonnées

Littérature citée [23 références]  Voir  Masquer  Télécharger

https://hal.archives-ouvertes.fr/hal-01350102
Contributeur : Laurent Besacier <>
Soumis le : vendredi 29 juillet 2016 - 16:09:34
Dernière modification le : jeudi 11 octobre 2018 - 08:48:03
Document(s) archivé(s) le : dimanche 30 octobre 2016 - 11:40:24

Fichier

metrics_correlation_asr-smt.pd...
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01350102, version 1

Collections

Citation

Ngoc-Tien Le, Christophe Servan, Benjamin Lecouteux, Laurent Besacier. Better Evaluation of ASR in Speech Translation Context Using Word Embeddings. Interspeech 2016, Sep 2016, San-Francisco, United States. Interspeech 2016 proceedings. 〈hal-01350102〉

Partager

Métriques

Consultations de la notice

479

Téléchargements de fichiers

293