Binding and unbinding the McGurk effect in audiovisual speech fusion: Follow-up experiments on a new paradigm

Olha Nahorna (1) , Frédéric Berthommier (1) , Jean-Luc Schwartz (1)

1 GIPSA-PCMD - GIPSA - Perception, Contrôle, Multimodalité et Dynamiques de la parole

The McGurk effect demonstrates the existence of a fusion process in audiovisual speech perception: the combination of the sound "ba" with the face of a speaker who pronounces "ga" is frequently perceived as "da". We assume that in the upstream of this phonetic fusion process, there is a "binding" process, which controls the combination of image and sound, and can block or reduce it in the case of audiovisual incoherencies (conditional binding process), as in the case of a dubbed film. To test and explore this binding hypothesis, we designed various experiments in which a coherent or incoherent audiovisual context is placed before McGurk stimuli, and we show that the incoherent contextual stimulus can significantly reduce the McGurk effect.

Mots clés: McGurk effect; binding; multisensory fusion; audiovisual speech perception; audiovisual scene analysis

Domaines: Neurosciences [q-bio.NC]

Olha Nahorna, Frédéric Berthommier, Jean-Luc Schwartz. Binding and unbinding the McGurk effect in audiovisual speech fusion: Follow-up experiments on a new paradigm. AVSP 2011 - 10th International Conference on Auditory-Visual Speech Processing, Aug 2011, Volterra, Italy. pp.21-24. ⟨hal-00642796⟩