An Articulatory-Based Singing Voice Synthesis Using Tongue and Lips Imaging

Abstract : Ultrasound imaging of the tongue and videos of lips movements can be used to investigate specific articulation in speech or singing voice. In this study, tongue and lips image sequences recorded during singing performance are used to predict vocal tract properties via Line Spectral Frequencies (LSF). We focused our work on traditional Corsican singing " Cantu in paghjella ". A multimodal Deep Autoencoder (DAE) extracts salient descriptors directly from tongue and lips images. Afterwards, LSF values are predicted from the most relevant of these features using a multilayer perceptron. A vocal tract model is derived from the predicted LSF, while a glottal flow model is computed from a synchronized electroglottographic recording. Articulatory-based singing voice synthesis is developed using both models. The quality of the prediction and singing voice synthesis using this method outperforms the state of the art method.
Type de document :
Communication dans un congrès
ISCA Interspeech 2016, Sep 2016, San Francisco, United States. Interspeech 2016, 2016, pp.1467 - 1471, 2016, 〈10.21437/Interspeech.2016-385〉
Liste complète des métadonnées

Littérature citée [13 références]  Voir  Masquer  Télécharger

https://hal.archives-ouvertes.fr/hal-01529630
Contributeur : Aurore Jaumard-Hakoun <>
Soumis le : mercredi 31 mai 2017 - 10:42:27
Dernière modification le : dimanche 4 juin 2017 - 01:07:28
Document(s) archivé(s) le : mercredi 6 septembre 2017 - 14:32:30

Fichier

IS16_AJaumard-Hakoun_revised.p...
Fichiers produits par l'(les) auteur(s)

Identifiants

Collections

Citation

Aurore Jaumard-Hakoun, Kele Xu, Clémence Leboullenger, Pierre Roussel-Ragot, Bruce Denby. An Articulatory-Based Singing Voice Synthesis Using Tongue and Lips Imaging. ISCA Interspeech 2016, Sep 2016, San Francisco, United States. Interspeech 2016, 2016, pp.1467 - 1471, 2016, 〈10.21437/Interspeech.2016-385〉. 〈hal-01529630〉

Partager

Métriques

Consultations de la notice

78

Téléchargements de fichiers

51