Sonifying drawings: characterization of perceptual attributes of sounds produced by human gestures

Etienne Thoret 1 Mitsuko Aramaki 1, * Richard Kronland-Martinet 1 Jean-Luc Velay 2 Solvi Ystad 1
* Corresponding author
1 Sons
LMA - Laboratoire de Mécanique et d'Acoustique [Marseille] : UPR7051
Abstract : Friction sounds produced by the pencil of a person who is drawing on a paper are audible and could bring information about his/her gestures. This study firstly focuses on the perceptual significance of the morphology of such sounds, and to what extent gestures could be retrieved by sounds. Sounds recorded during drawing sessions were used in association tests where subjects had to univocally associate friction sounds with different shapes. Results showed that subjects were able to associate sounds with correct shapes and that the auditory characterization of the shape depended on the velocity profile.\\ A sonification strategy was then proposed for human drawings using a friction sound synthesis model. Inspired by the work of Viviani et al. (1982) who exhibited a 2/3-power law relation between kinetics and shape curvature for visual perception, experiments were carried out with the synthesis model, where subjects were asked to adjust the power law exponent so that the most realistic sound was obtained. Results revealed an exponent close to 2/3 as previously found in vision and thus highlighted a similar power law in the auditory modality, providing an ecological way to determine velocity profiles from static shapes and to generate sounds coherent with human gestures.
Complete list of metadatas

Cited literature [13 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-00811244
Contributor : Hal System <>
Submitted on : Monday, April 23, 2012 - 10:00:00 AM
Last modification on : Monday, March 4, 2019 - 2:04:25 PM
Long-term archiving on : Sunday, December 18, 2016 - 11:20:41 AM

File

hal-00811244.pdf
Publisher files allowed on an open archive

Identifiers

  • HAL Id : hal-00811244, version 1

Citation

Etienne Thoret, Mitsuko Aramaki, Richard Kronland-Martinet, Jean-Luc Velay, Solvi Ystad. Sonifying drawings: characterization of perceptual attributes of sounds produced by human gestures. Acoustics 2012, Apr 2012, Nantes, France. ⟨hal-00811244⟩

Share

Metrics

Record views

377

Files downloads

199