From 3-D speaker cloning to text-to-audiovisual speech

Sascha Fagel (1) , Frédéric Elisei (2) , Gérard Bailly (3)

1 ISK - Institut für Sprache und Kommunikation
2 GIPSA-Services - GIPSA-Services
3 GIPSA-MPACIF - GIPSA - Machines Parlantes, Agents Communicants & Interaction Face-à-face

Visible speech movements were motion captured and parameterized. Coarticulated targets were extracted from VCVs and modeled to generate arbitrary German utterances by target interpolation. The system was extended to synthesize English utterances by a mapping to German phonemes. An evaluation by means of a modified rhyme test reveals that the synthetic videos of isolated words increase the recognition scores from 27 % to 47.5 % when added to audio only presentation

Domains: Signal and Image processing; Signal and Image Processing

Sascha Fagel, Frédéric Elisei, Gérard Bailly. From 3-D speaker cloning to text-to-audiovisual speech. Interspeech 2008 - 9th Annual Conference of the International Speech Communication Association, Sep 2008, Brisbane, Australia. pp.2325. ⟨hal-00361886⟩