Descriptor-based Sound Texture Sampling

Diemo Schwarz (1, 2) , Norbert Schnell (1, 2)

1 Equipe Interactions musicales temps-réel
2 Equipe Autre (hors R&D)

Existing methods for sound texture synthesis are often concerned with the extension of a given recording, while keeping its overall properties and avoiding artefacts. However, they generally lack controllability of the resulting sound texture. After a review and classification of existing approaches, we propose two methods of statistical modeling of the audio descriptors of texture recordings using histograms and Gaussian mixture models. The models can be interpolated to steer the evolution of the sound texture between different target recordings (e.g. from light to heavy rain). Target descriptor values are stochastically drawn from the statistic models by inverse transform sampling to control corpus-based concatenative synthesis for the final sound generation, that can also be controlled interactively by navigation through the descriptor space. To better cover the target descriptor space, we expand the corpus by automatically generating variants of the source sounds with transformations applied, and storing only the resulting descriptors and the transformation parameters in the corpus.

Mots clés: concatenative synthesis; Gaussian Mixture Models; databases; content-based retrieval; corpus-based synthesis; sound textures; statistic modeling; audio descriptors

Mots clés: Informatique musicale

Domaines: Son [cs.SD]; Interface homme-machine [cs.HC]; Musique, musicologie et arts de la scène; Informatique [cs]; Traitement du signal et de l'image [eess.SP]; Apprentissage [cs.LG]; Intelligence artificielle [cs.AI]; Ingénierie assistée par ordinateur; Multimédia [cs.MM]; Vision par ordinateur et reconnaissance de formes [cs.CV]; Autre [cs.OH]; Traitement du signal et de l'image [eess.SP]

Diemo Schwarz, Norbert Schnell. Descriptor-based Sound Texture Sampling. Sound and Music Computing (SMC), Jul 2010, Barcelona, Spain. pp.510-515. ⟨hal-01161251⟩