Deep learning for musical scenario inference and prediction: Application to structured co-improvisation

Théis Bazin 1, 2
1 Repmus - Représentations musicales
STMS - Sciences et Technologies de la Musique et du Son
Abstract : The field of musical scenario inference aims at developing systems and algorithms to automatically extract abstract temporal scenarios in music. We call scenario any underlying symbolic sequence that constitutes a higher-level abstraction of an original input sequence. Such an underlying sequence implicitly encodes the temporal relations between events in a musical piece by producing an ordered series of symbols. Musical works exhibit temporal dependencies at multiple time-scales, from local melodic events to long-term harmonic progressions. Multiple systems have been introduced in order to capture short or long term dependencies between musical events. Nonetheless, existing systems fail at taking into account the interactions between these various time scales. In this research project, we propose a method to tackle this issue and infer abstract scenarios through the use of deep recurrent neural networks. We introduce a system that is able to extract an abstract sequence of symbols from an input musical sequence, as well as perform predictions on the probable continuations of this sequence. A theoretical application to the co-improvisation problem is introduced. Co-improvisation engines seek to generate new sequences resembling some example input sequence. A crucial aspect of such co-improvisation systems is the ability to introduce anticipations, so as to generate transitions between different parts. This requires knowledge of some underlying scenario to the generation. Existing systems that offer prediction capacities rely on a pre-defined abstract scenario. The architecture we propose would improve on this by replacing this pre-defined scenario with one automatically inferred in real-time by our scenario inference and prediction tool, incorporating dynamically refined short-term predictions over the future. Through dynamic training via adversarial training, this system can furthermore improve the accuracy of its predictions in real-time.
Liste complète des métadonnées
Contributor : Jérôme Nika <>
Submitted on : Thursday, October 27, 2016 - 12:51:36 PM
Last modification on : Thursday, March 21, 2019 - 2:16:14 PM


  • HAL Id : hal-01388645, version 1


Théis Bazin. Deep learning for musical scenario inference and prediction: Application to structured co-improvisation. [Intership report] Ircam UMR STMS 9912. 2016. ⟨hal-01388645⟩



Record views