DALI: A Large Dataset Of Synchronized Audio, Lyrics And Notes, Automatically Created Using Teacher-student Machine Learning Paradigm

Gabriel Meseguer-Brocal 1 Alice Cohen-Hadria 2 Geoffroy Peeters 3
1 Analyse et synthèse sonores;
STMS - Sciences et Technologies de la Musique et du Son, IRCAM - Institut de Recherche et Coordination Acoustique/Musique
2 Analyse et synthèse sonores [Paris]
STMS - Sciences et Technologies de la Musique et du Son
Abstract : The goal of this paper is twofold. First, we introduce DALI, a large and rich multimodal dataset containing 5358 audio tracks with their time-aligned vocal melody notes and lyrics at four levels of granularity. The second goal is to explain our methodology where dataset creation and learning models interact using a teacher-student machine learning paradigm that benefits each other. We start with a set of manual annotations of draft time-aligned lyrics and notes made by non-expert users of Karaoke games. This set comes without audio. Therefore, we need to find the corresponding audio and adapt the annotations to it. To that end, we retrieve audio candidates from the Web. Each candidate is then turned into a singing-voice probability over time using a teacher, a deep convolutional neural network singing-voice detection system (SVD), trained on cleaned data. Comparing the time-aligned lyrics and the singing-voice probability, we detect matches and update the time-alignment lyrics accordingly. From this, we obtain new audio sets. They are then used to train new SVD students used to perform again the above comparison. The process could be repeated iteratively. We show that this allows to progressively improve the performances of our SVD and get better audio-matching and alignment.
Complete list of metadatas

Cited literature [28 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-02019115
Contributor : Gabriel Meseguer Brocal <>
Submitted on : Thursday, February 14, 2019 - 12:38:57 PM
Last modification on : Monday, January 27, 2020 - 6:00:28 PM
Long-term archiving on: Wednesday, May 15, 2019 - 7:07:20 PM

File

35_Paper.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02019115, version 1

Citation

Gabriel Meseguer-Brocal, Alice Cohen-Hadria, Geoffroy Peeters. DALI: A Large Dataset Of Synchronized Audio, Lyrics And Notes, Automatically Created Using Teacher-student Machine Learning Paradigm. 19th International Society for Music Information Retrieval Conference, Sep 2018, Paris, France. ⟨hal-02019115⟩

Share

Metrics

Record views

54

Files downloads

66