Audio-visual speech scene analysis: Characterization of the dynamics of unbinding and rebinding the McGurk effect

Olha Nahorna 1 Frédéric Berthommier 1 Jean-Luc Schwartz 1
1 GIPSA-PCMD - PCMD
GIPSA-DPC - Département Parole et Cognition
Abstract : While audiovisual interactions in speech perception have long been considered as automatic, recent data suggest that this is not the case. In a previous study, Nahorna et al. [(2012). J. Acoust. Soc. Am. 132, 1061–1077] showed that the McGurk effect is reduced by a previous incoherent audiovisual context. This was interpreted as showing the existence of an audiovisual binding stage controlling the fusion process. Incoherence would produce unbinding and decrease the weight of the visual input in fusion. The present paper explores the audiovisual binding system to characterize its dynamics. A first experiment assesses the dynamics of unbinding, and shows that it is rapid: An incoherent context less than 0.5 s long (typically one syllable) suffices to produce a maximal reduction in the McGurk effect. A second experiment tests the rebinding process, by presenting a short period of either coherent material or silence after the incoherent unbinding context. Coherence provides rebinding, with a recovery of the McGurk effect, while silence provides no rebinding and hence freezes the unbinding process. These experiments are interpreted in the framework of an audiovisual speech scene analysis process assessing the perceptual organization of an audiovisual speech input before decision takes place at a higher processing stage.
Type de document :
Article dans une revue
Journal of the Acoustical Society of America, Acoustical Society of America, 2015, 137 (1), pp.362-377. 〈10.1121/1.4904536〉
Liste complète des métadonnées

Littérature citée [46 références]  Voir  Masquer  Télécharger

https://hal.archives-ouvertes.fr/hal-01213897
Contributeur : Jean-Luc Schwartz <>
Soumis le : vendredi 9 octobre 2015 - 17:03:03
Dernière modification le : lundi 9 avril 2018 - 12:22:49
Document(s) archivé(s) le : dimanche 10 janvier 2016 - 10:23:53

Fichier

Nahorna_JASA_Binding2_second_r...
Fichiers produits par l'(les) auteur(s)

Identifiants

Collections

Citation

Olha Nahorna, Frédéric Berthommier, Jean-Luc Schwartz. Audio-visual speech scene analysis: Characterization of the dynamics of unbinding and rebinding the McGurk effect. Journal of the Acoustical Society of America, Acoustical Society of America, 2015, 137 (1), pp.362-377. 〈10.1121/1.4904536〉. 〈hal-01213897〉

Partager

Métriques

Consultations de la notice

2098

Téléchargements de fichiers

153