Robust Articulatory Speech Synthesis using Deep Neural Networks for BCI Applications

Florent Bocquelet (1, 2) , Thomas Hueber (1) , Laurent Girin (1) , Pierre Badin (1) , Blaise Yvert (2)

1 GIPSA-MAGIC - GIPSA - Machines parlantes, Gestes oro-faciaux, Interaction Face-à-face, Communication augmentée
2 SCLIN - Clinatec - Centre de recherche biomédicale Edmond J.Safra

Brain-Computer Interfaces (BCIs) usually propose typing strategies to restore communication for paralyzed and aphasic people. A more natural way would be to use speech BCI directly controlling a speech synthesizer. Toward this goal, a prerequisite is the development a synthesizer that should i) produce intelligible speech, ii) run in real time, iii) depend on as few parameters as possible, and iv) be robust to error fluctuations on the control parameters. In this context, we describe here an articulatory-to-acoustic mapping approach based on deep neural network (DNN) trained on electromagnetic articulography (EMA) data recorded synchronously with produced speech sounds. On this corpus, the DNN-based model provided a speech synthesis quality (as assessed by automatic speech recognition and behavioral testing) comparable to a state-of-the-art Gaussian mixture model (GMM), yet showing higher robustness when noise was added to the EMA coordinates. Moreover, to envision BCI applications, this robustness was also assessed when the space covered by the 12 original articulatory parameters was reduced to 7 parameters using deep auto-encoders (DAE). Given that this method can be implemented in real time, DNN-based articulatory speech synthesis seems a good candidate for speech BCI applications.

Mots clés: dimensionality reduction; deep neural networks; deep auto-encoder; EMA; noise robustness; brain computer interface (BCI); articulatory speech synthesis

Domaines: Sciences de l'information et de la communication

Florent Bocquelet, Thomas Hueber, Laurent Girin, Pierre Badin, Blaise Yvert. Robust Articulatory Speech Synthesis using Deep Neural Networks for BCI Applications. Interspeech 2014 - 15th Annual Conference of the International Speech Communication Association, Sep 2014, Singapour, Singapore. ⟨hal-01228891⟩