Glottal Flow Synthesis for Whisper-to-Speech Conversion

Olivier Perrotin (1) , Ian V. Mcloughlin (2)

1 GIPSA-CRISSP - GIPSA - Cognitive Robotics, Interactive Systems, & Speech Processing
2 University of Kent [Canterbury]

Whisper-to-speech conversion is motivated by laryngeal disorders, in which malfunction of the vocal folds leads to loss of voicing. Many patients with laryngeal disorders can still produce functional whispers, since these are characterised by the absence of vocal fold vibration. Whispers therefore constitute a common ground for speech rehabilitation across many kinds of laryngeal disorder. Whisper-to-speech conversion involves recreating natural-sounding speech from recorded whispers, and is a non-invasive and non-surgical rehabilitation that can maintain a natural method of speaking, unlike the existing methods of rehabilitation. This paper proposes a new rule-based method for whisper-to-speech conversion that replaces the noisy whisper sound source with a synthesised speech-like harmonic source, while maintaining the vocal tract component unaltered. In particular, a novel glottal source generator is developed in which whisper information is used to parameterise the excitation through a high-quality glottis model. Evaluation of the system against the standard pulse train excitation method reveals significantly improved performance. Since our method is glottis-based, it is potentially compatible with the many existing vocal tract component adaptation systems.

Mots clés: Whisper-to-speech conversion; Speech synthesis; Vocal source excitation; Glottal Flow Model; Laryngectomy

Domaines: Interface homme-machine [cs.HC]; Traitement du signal et de l'image [eess.SP]

Olivier Perrotin, Ian V. Mcloughlin. Glottal Flow Synthesis for Whisper-to-Speech Conversion. IEEE/ACM Transactions on Audio, Speech and Language Processing, 2020, 28, pp.889-900. ⟨10.1109/TASLP.2020.2971417⟩. ⟨hal-02518246⟩