What Musical Knowledge Does Self-Attention Learn?

Mikaela Keller (1, 2) , Gabriel Loiseau (2) , Louis Bigo (3, 2)

1 MAGNET - Machine Learning in Information Networks
2 CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189
3 Algomus

Since their conception for NLP tasks in 2017, Transformer neural networks have been increasingly used with compelling results for a variety of symbolic MIR tasks including music analysis, classification and generation. Although the concept of self-attention between words in text can intuitively be transposed as a relation between musical objects such as notes or chords in a score, it remains relatively unknown what kind of musical relations precisely tend to be captured by self attention mechanisms when applied to musical data. Moreover, the principle of self-attention has been elaborated in NLP to help model the “meaning” of a sentence while in the musical domain this concept appears to be more subjective. In this explorative work, we open the music transformer black box looking to identify which aspects of music are actually learnt by the self-attention mechanism. We apply this approach to two MIR probing tasks : composer classification and cadence identification.

Domaines: Musique, musicologie et arts de la scène; Son [cs.SD]; Machine Learning [stat.ML]

Mikaela Keller, Gabriel Loiseau, Louis Bigo. What Musical Knowledge Does Self-Attention Learn?. Workshop on NLP for Music and Spoken Audio (NLP4MuSA 2021), 2021, Online, France. ⟨hal-03419236v2⟩