Predicting F0 and voicing from NAM-captured whispered speech

Viet-Anh Tran (1) , Gérard Bailly (2) , Hélène Loevenbruck (3) , Tomoki Toda (4)

1 GIPSA-lab - Grenoble Images Parole Signal Automatique
2 GIPSA-MPACIF - GIPSA - Machines Parlantes, Agents Communicants & Interaction Face-à-face
3 GIPSA-PMD - GIPSA - Parole, Multimodalité, Développement
4 SAPL - Speech and Acoustics Processing Laboratory

The NAM-to-speech conversion proposed by Toda and colleagues which converts Non-Audible Murmur (NAM) to audible speech by statistical mapping trained using aligned corpora is a very promising technique, but its performance is still insufficient, mainly due to the difficulty in estimating F0 of the transformed voice from unvoiced speech. In this paper, we propose a method to improve F0 estimation and voicing decision in a NAM-to-speech conversion system based on Gaussian Mixture Models (GMM) applied to whispered speech. Instead of combining voicing decision and F0 estimation in a single GMM, a simple feed-forward neural network is used to detect voiced segments in the whisper while a GMM estimates a continuous melodic contour based on training voiced segments. The error rate for the voiced/unvoiced decision of the network is 6.8% compared to 9.2% with the original system. Our proposal benefits also to F0 estimation error.

Domaines: Traitement du signal et de l'image [eess.SP]; Traitement du signal et de l'image [eess.SP]

Viet-Anh Tran, Gérard Bailly, Hélène Loevenbruck, Tomoki Toda. Predicting F0 and voicing from NAM-captured whispered speech. Speech Prosody 2008 - 4th International Conference on Speech Prosody, May 2008, Campinas, Brazil. pp.107-110. ⟨hal-00333290⟩