Improved Neural Bag-of-Words Model to Retrieve Out-of-Vocabulary Words in Speech Recognition

Imran Sheikh (1) , Irina Illina (1) , Dominique Fohr (1) , Georges Linares (2)

1 MULTISPEECH - Speech Modeling for Facilitating Oral-Based Communication
2 LIA - Laboratoire Informatique d'Avignon

Many Proper Names (PNs) are Out-Of-Vocabulary (OOV) words for speech recognition systems used to process di-achronic audio data. To enable recovery of the PNs missed by the system, relevant OOV PNs can be retrieved by exploiting the semantic context of the spoken content. In this paper, we explore the Neural Bag-of-Words (NBOW) model, proposed previously for text classification, to retrieve relevant OOV PNs. We propose a Neural Bag-of-Weighted-Words (NBOW2) model in which the input embedding layer is augmented with a context anchor layer. This layer learns to assign importance to input words and has the ability to capture (task specific) keywords in a NBOW model. With experiments on French broadcast news videos we show that the NBOW and NBOW2 models outper-form earlier methods based on raw embeddings from LDA and Skip-gram. Combining NBOW with NBOW2 gives faster convergence during training.

Mots clés: lvcsr; oov; proper names

Domaines: Interface homme-machine [cs.HC]

Imran Sheikh, Irina Illina, Dominique Fohr, Georges Linares. Improved Neural Bag-of-Words Model to Retrieve Out-of-Vocabulary Words in Speech Recognition. INTERSPEECH 2016, Sep 2016, San Francisco, United States. ⟨10.21437/Interspeech.2016-1219⟩. ⟨hal-01384488⟩