Analysis and synthesis of the 3D movements of the head, face and hand of a speaker using cued speech - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Journal of the Acoustical Society of America Année : 2005

Analysis and synthesis of the 3D movements of the head, face and hand of a speaker using cued speech

Résumé

This paper present here efforts for characterizing the 3D movements of the right hand and the face of a French female during the production of cued speech. The 3D trajectories of 50 hand and 63 facial fleshpoints during the production of 238 utterances are analyzed. These utterances are carefully designed to cover all possible diphones of French. Linear and nonlinear statistical models of the deformations and the postures of the hand and the face have been developed using separate and joint corpora. Recognition of hand and face postures at targets is performed to verify a posteriori that key hand movements and postures imposed by cued speech had been well realized by the subject. Recognition results are further exploited in order to study the phonetic structure of cued speech notably the phasing relations between hand gestures and sound production. A first implementation of a concatenative audiovisual text-to-cued speech synthesis system is finally described that employs this unique and extensive data on cued speech in action.

Domaines

Informatique
Fichier principal
Vignette du fichier
gb_JASA05.pdf (936.27 Ko) Télécharger le fichier
Origine : Accord explicite pour ce dépôt
Loading...

Dates et versions

hal-00143622 , version 1 (26-04-2007)

Identifiants

  • HAL Id : hal-00143622 , version 1

Citer

Guillaume Gibert, Gérard Bailly, Denis Beautemps, Frédéric Elisei. Analysis and synthesis of the 3D movements of the head, face and hand of a speaker using cued speech. Journal of the Acoustical Society of America, 2005, 118 (2), pp.1144-1153. ⟨hal-00143622⟩

Collections

UGA CNRS ICP
113 Consultations
289 Téléchargements

Partager

Gmail Facebook X LinkedIn More