FOLDED CQT RCNN FOR REAL-TIME RECOGNITION OF INSTRUMENT PLAYING TECHNIQUES

Jean-Francois Ducher (1, 2, 3) , Philippe Esling (3)

1 CICM - Centre de recherche Informatique et Création Musicale
2 MUSIDANSE - Esthétique, musicologie, danse et création musicale
3 Repmus - Représentations musicales

In the past years, deep learning has produced state-of-the-art performance in timbre and instrument classification. However, only a few models currently deal with the recognition of advanced Instrument Playing Techniques (IPT). None of them have a real-time approach of this problem. Furthermore, most studies rely on a single sound bank for training and testing. Their methodology provides no assurance as to the generalization of their results to other sounds. In this article, we extend state-of-the-art convolutional neural networks to the classification of IPTs. We build the first IPT corpus from independent sound banks, annotate it with the JAMS standard and make it freely available. Our models yield consistently high accuracies on a homogeneous subset of this corpus. However, only a proper taxonomy of IPTs and specifically defined input transforms offer proper resilience when addressing the "minus-1db" methodology, which assesses the ability of the models to generalize. In particular, we introduce a novel Folded Constant Q-Transform adjusted to the requirements of IPT classification. Finally we discuss the use of our classifier in real-time.

Domaines: Intelligence artificielle [cs.AI]; Musique, musicologie et arts de la scène; Son [cs.SD]; Traitement du signal et de l'image [eess.SP]; Réseau de neurones [cs.NE]; Multimédia [cs.MM]; Apprentissage [cs.LG]

Jean-Francois Ducher, Philippe Esling. FOLDED CQT RCNN FOR REAL-TIME RECOGNITION OF INSTRUMENT PLAYING TECHNIQUES. International Society for Music Information Retrieval, Nov 2019, Delft, Netherlands. ⟨hal-02472560⟩