Auditory perception of the actions causing the sounds : a basis for the design of sonic interactions
Résumé
Research in sound perception generally focuses on a few specific auditory attributes (pitch, loudness, sharpness, roughness, etc.) and their corresponding acoustic properties. Similarly, a common approach in data and interface sonification is to map such attributes to different pieces of information to be decoded by the listener. However, when asked to report what they hear, lay listeners spontaneously describe what has caused the sound (i.e. the sound sources) rather than some attributes of the sound itself. Even highly-trained sound experts encounter difficulties to abstract a sound from its source and focus on acoustics properties, when they recognize the sound sources. This distinction between musical listening (focusing on acoustic properties) and everyday listening (identifying the sources of the sounds) was formalized by seminal sound design researcher William Gaver, who for instance proposed to rely on the semantics of identified everyday sounds (e.g. throwing something to the garbage can) in his design of the sounds of an interface for Apple computers. But what do we perceive exactly from the sound sources ? I will argue in this presentation that auditory perception may actually be better suited for the perception of the actions and events that cause the sounds rather than the properties of the objects set in vibration. For instance, despite auditory perception of material has been extensively studied, results show that material identification is only moderately accurate and only in certain circumstances. On the contrary identification of different actions causing the sounds (e.g. tapping, scraping) is always accurate, no matter the objects on which the actions are executed. Recent research shows that the binding of auditory perception and action execution is even more pronounced. Studies in neuroscience has recently shown that listening to action sounds (and in particular the sounds of tool manipulation) activates brain area involved in motor planning and execution. At a behavioral level, we have shown that listening to a sound associated with an action can prime executing that action. These results are expected to provide a basis for the design sonic interaction and sonification in a number of domains such as sport training and motor rehabilitation. For instance, I will report results that show that adding a complex sonic feedback to a tangible interface allowed users to learn how to adjust the fine parameters a target gesture and better control the interface.