Skip to Main content Skip to Navigation
Conference papers

End-to-end model for named entity recognition from speech without paired training data

Abstract : Recent works showed that end-to-end neural approaches tend to become very popular for spoken language understanding (SLU). Through the term end-to-end, one considers the use of a single model optimized to extract semantic information directly from the speech signal. A major issue for such models is the lack of paired audio and textual data with semantic annotation. In this paper, we propose an approach to build an end-to-end neural model to extract semantic information in a scenario in which zero paired audio data is available. Our approach is based on the use of an external model trained to generate a sequence of vectorial representations from text. These representations mimic the hidden representations that could be generated inside an end-to-end automatic speech recognition (ASR) model by processing a speech signal. A SLU neural module is then trained using these representations as input and the annotated text as output. Last, the SLU module replaces the top layers of the ASR model to achieve the construction of the end-to-end model. Our experiments on named entity recognition, carried out on the QUAERO corpus, show that this approach is very promising, getting better results than a comparable cascade approach or than the use of synthetic voices.
Document type :
Conference papers
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-03701145
Contributor : Yannick Estève Connect in order to contact the contributor
Submitted on : Tuesday, June 21, 2022 - 4:49:05 PM
Last modification on : Friday, July 1, 2022 - 3:32:15 AM

File

IS22___Textual_injection_for_e...
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03701145, version 1

Collections

Citation

Salima Mdhaffar, Jarod Duret, Titouan Parcollet, Yannick Estève. End-to-end model for named entity recognition from speech without paired training data. Interspeech 2022, Sep 2022, Incheon, South Korea. ⟨hal-03701145⟩

Share

Metrics

Record views

2

Files downloads

0