PiPo, A Plugin Interface for Afferent Data Stream Processing Modules

Norbert Schnell (1) , Diemo Schwarz (1) , Joseph Larralde (1) , Riccardo Borghesi (1)

1 Interaction Son Musique Mouvement [Paris]

We present PiPo, a plugin API for data stream processing with applications in interactive audio processing and music information retrieval as well as potentially other domains of signal processing. The development of the API has been motivated by our recurrent need to use a set of signal processing modules that extract low-level descriptors from audio and motion data streams in the context of different au-thoring environments and end-user applications. The API is designed to facilitate both, the development of modules and the integration of modules or module graphs into applications. It formalizes the processing of streams of multidimensional data frames which may represent regularly sampled signals as well as time-tagged events or numeric annotations. As we found it sufficient for the processing of incoming (i.e. afferent) data streams, PiPo modules have a single input and output and can be connected to sequential and parallel processing paths. After laying out the context and motivations, we present the concept and implementation of the PiPo API with a set of modules that allow for extracting low-level descriptors from audio streams. In addition, we describe the integration of the API into host environments such as Max, Juce, and OpenFrameworks.

Mots clés: data flow; audio analysis; gesture analysis; movement; motion; API; plugin

Domaines: Son [cs.SD]; Algorithme et structure de données [cs.DS]; Multimédia [cs.MM]

Norbert Schnell, Diemo Schwarz, Joseph Larralde, Riccardo Borghesi. PiPo, A Plugin Interface for Afferent Data Stream Processing Modules. International Symposium on Music Information Retrieval (ISMIR), Oct 2017, Suzhou, China. ⟨hal-01575288⟩