Music Generation by Deep Learning - Challenges and Directions

Jean-Pierre Briot 1 Francois Pachet
1 SMA - Systèmes Multi-Agents
LIP6 - Laboratoire d'Informatique de Paris 6
Abstract : In addition to traditional tasks such as prediction, classification and translation, deep learning is receiving growing attention as an approach for music generation, as witnessed by recent research groups such as Magenta at Google and CTRL (Creator Technology Research Lab) at Spotify. The motivation is in using the capacity of deep learning architectures and training techniques to automatically learn musical styles from arbitrary musical corpora and then to generate samples from the estimated (learnt) distribution. Meanwhile, a direct application of deep learning generation, such as feedforward on a feedforward or a recurrent architecture, reaches some limits as they tend to mimic the corpus learnt without incentive for creativity. Moreover, deep learning architectures do not offer direct ways for controlling generation (e.g., imposing some tonality or other arbitrary constraints). Furthermore, deep learning architectures alone are autistic automata which generate music autonomously without human user interaction, far from the objective of assisting musicians to compose and refine music. Issues such as: control, creativity and interaction are the focus of our analysis. In this paper, we list various limitations of a direct application of deep learning to music generation, analyze why the issues are not fulfilled and how to address them by possible approaches. Various examples of recent systems are cited as examples of promising directions.
Complete list of metadatas

Cited literature [56 references]  Display  Hide  Download
Contributor : Jean-Pierre Briot <>
Submitted on : Monday, December 11, 2017 - 1:17:34 PM
Last modification on : Thursday, March 21, 2019 - 2:19:20 PM


Files produced by the author(s)


Distributed under a Creative Commons Attribution 4.0 International License


  • HAL Id : hal-01660753, version 1


Jean-Pierre Briot, Francois Pachet. Music Generation by Deep Learning - Challenges and Directions. 2017. ⟨hal-01660753⟩



Record views


Files downloads