“Listen to the Picture!”. The StatSon Sound Sonification System, using VST and DSP

Martin Heath , Gordon Hunter

StatSon is a proof-of-concept C++ plugin for a VST host (for example, the Cubase music production software suite) using digital signal processing (DSP). Its inputs are an audio signal that is converted to the frequency domain using a phase vocoder and an image that is analysed statistically. Selected statistics for the image can become pitch manipulation parameters controlling various transformations of the input signal’s loudest frequency. Arbitrary input can be rounded to the closest semitone, scalar tone or tone from a selection of three or four note chords, allowing integration with existing music. A wavetable synthesiser converts the transformed frequency to a monophonic sinusoidal output, changing typically every 10ms, producing an outcome which is sometimes reminiscent of granular synthesis. The output is able to track the input and change recognisably in response to changing the image. The modified audio signal can therefore be regarded as a primitive “sonification” of the current image. The aim of the work was to create an audio effect for use live or in the studio, whilst exploring this technology and the field of image sonification.

Mots clés: Sonification; Statistics; DSP; VST

Domaines: Acoustique [physics.class-ph]

Martin Heath, Gordon Hunter. “Listen to the Picture!”. The StatSon Sound Sonification System, using VST and DSP. Acoustics 2012, Apr 2012, Nantes, France. ⟨hal-00810585⟩