Analysis and synthesis of the 3D movements of the head, face and hand of a speaker using cued speech
Résumé
This paper present here efforts for characterizing the 3D movements of the right hand and the face of a French female during the production of cued speech. The 3D trajectories of 50 hand and 63 facial fleshpoints during the production of 238 utterances are analyzed. These utterances are carefully designed to cover all possible diphones of French. Linear and nonlinear statistical models of the deformations and the postures of the hand and the face have been developed using separate and joint corpora. Recognition of hand and face postures at targets is performed to verify a posteriori that key hand movements and postures imposed by cued speech had been well realized by the subject. Recognition results are further exploited in order to study the phonetic structure of cued speech notably the phasing relations between hand gestures and sound production. A first implementation of a concatenative audiovisual text-to-cued speech synthesis system is finally described that employs this unique and extensive data on cued speech in action.
Domaines
Informatique
Origine : Accord explicite pour ce dépôt
Loading...