Integration of auditory, labial and manual signals in cued speech perception by deaf adults : an adaptation of the McGurk paradigm

Clémence Bayard (1, 2) , Jacqueline Leybaert (2) , Cécile Colin (2)

1 GIPSA-PCMD - GIPSA - Perception, Contrôle, Multimodalité et Dynamiques de la parole
2 ULB/CRCN - Center for Research in Cognition & Neurosciences [Brussels]

Among deaf individuals fitted with a cochlear implant, some use Cued Speech (CS; a system in which each syllable is uttered with a complementary manual gesture) and therefore, have to combine auditory, labial and manual information to perceive speech. We examined how audio-visual (AV) speech integration is affected by the presence of manual cues and on which form of information (auditory, labial or manual) the CS receptors primarily rely depending on labial ambiguity. To address this issue, deaf CS users (N=36) and deaf CS naïve (N=35) participants were submitted to an identification task of two AV McGurk stimuli (either with a plosive or with a fricative consonant). Manual cues were congruent with either auditory information, lip information or the expected fusion. Results revealed that deaf individuals can merge audio and labial information into a single unified percept. Without manual cues, participants gave a high proportion of fusion response (particularly with ambiguous plosive McGurk stimuli). Results also suggested that manual cues can modify the AV integration and that their impact differs between plosive and fricative McGurk stimuli.

Domaines: Psychologie

Clémence Bayard, Jacqueline Leybaert, Cécile Colin. Integration of auditory, labial and manual signals in cued speech perception by deaf adults : an adaptation of the McGurk paradigm. FAAVSP 2015 - 1st Joint Conference on Facial Analysis, Animation and Auditory-Visual Speech Processing, Sep 2015, Vienne, Austria. ⟨hal-01302422⟩