Multisensory integration in percussion performance

Bruno L. Giordano (1) , Federico Avanzini (2) , Marcelo M. Wanderley (3) , Stephen Mcadams (1)

1 CIRMMT - Centre Interdisciplinaire de Recherche en Musique, Médias et Technologie [Montréal]
2 DEI - Department of Information Engineering [Padova]
3 McGill University = Université McGill [Montréal, Canada]

We investigated how auditory and haptic information concerning objects hardness is integrated for the purpose of controlling the velocity with which we strike an object. Our experimental manipulations and data analyses considered a variety of factors that should be integrated in a theory of multisensory perception: expertise of the perceiver; context (unimodal vs. multimodal); inter-modality congruence; inter-participants agreement in sensory weighting; performance. On each trial, participants struck a virtual object with a constant target velocity and received feedback on correctness. When the performance criterion was reached, feedback was eliminated, the auditory and/or haptic hardness of the struck object were changed, and the e ects on subsequent striking velocity and performance were measured. In unimodal trials only the haptic or auditory display was presented. In multisensory trials, the audio-haptic changes could be congruent (e.g., both increased in hardness) or incongruent. We recruited participants with di erent levels of expertise with the task: percussionists, nonpercussionist musicians and nonmusicians. For both modalities, striking velocity increased with decreasing hardness, and vice versa. With the vast majority of participants, changes in haptic hardness were perceptually more relevant because they in- uenced striking velocity to a greater degree than did changes in auditory hardness. The perceptual weighting of auditory information was robust to context variations (unimodal vs. multimodal), independent of expertise, uniform across participants and modulated by audio-haptic congruence. The perceptual weighting of haptic information was modulated by context and expertise, was more varied across participants and was robust to changes in audio-haptic congruence. Performance in velocity tracking was more strongly a ected by haptic than by auditory information, was not at its best in a multisensory context and was independent of information congruence.

Mots clés: Intégration multisensorielle; Perception audio-haptique; Interactions geste-perception

Domaines: Acoustique [physics.class-ph]; Acoustique [physics.class-ph]

Bruno L. Giordano, Federico Avanzini, Marcelo M. Wanderley, Stephen Mcadams. Multisensory integration in percussion performance. 10ème Congrès Français d'Acoustique, Apr 2010, Lyon, France. ⟨hal-00542613⟩