Speech technologies for augmented communication

Gérard Bailly (1) , Pierre Badin (1) , Denis Beautemps (1) , Frédéric Elisei (2)

1 GIPSA-MAGIC - GIPSA - Machines parlantes, Gestes oro-faciaux, Interaction Face-à-face, Communication augmentée
2 GIPSA-Services - GIPSA-Services

We introduce here an emerging technological and scientific field. Augmented speech communication (ASC) aims at supplementing human-human communication with enhanced or additional modalities. ASC improves human-human communication by exploiting a priori knowledge on multimodal coherence of speech signals, user/listener voice characteristics or more general linguistic and phonological structure on the spoken language or vocabulary being exchanged. The nature of this a priori knowledge, the quantitative models that implement it and their capabilities to enhance the available input signals influence the precision and robustness of the perceived signals. After a general overview of the possible input signals characterizing speech production activity and available technologies for mapping these various speech representation between each other, three ASC systems developed at GIPSA-Lab are described in detail. Preliminary results of the evaluation of these three systems will be given and commented. A discussion on scientific and technological challenges and limitations of ASC concludes the chapter.

Mots clés: Augmented speech communication; Talking heads; Cued speech; Silent speech interfaces; Tongue reading; Voice conversion; Speech maps; Electromagnetoarticulography; Ultrasound imaging; Motion capture

Domaines: Traitement du signal et de l'image [eess.SP]; Traitement du signal et de l'image [eess.SP]

Gérard Bailly, Pierre Badin, Denis Beautemps, Frédéric Elisei. Speech technologies for augmented communication. J. Mullennix and S. Stern. Computer synthesized speech technologies: tools for aiding impairment, IGI Global, Hershey, PA, pp.116-128, 2010. ⟨hal-00473026⟩