Large Dimensional Asymptotics of Multi-Task Learning

Malik Tiomoko (1, 2) , Cosme Louart (2) , Romain Couillet (2)

1 CentraleSupélec
2 GIPSA-GAIA - GIPSA Pôle Géométrie, Apprentissage, Information et Algorithmes

Inspired by human learning, which transfers knowledge from learned tasks to solve new tasks, multitask learning aims at simultaneously solving multiple tasks by a smart exploitation of their similarities. How to relate the tasks so to optimize their performances is however a largely open problem. Based on a random matrix approach, this article proposes an asymptotic analysis of a support vector machine-inspired multitask learning scheme. The asymptotic performance of the algorithm, validated on both synthetic and real data, sets forth the relation between the statistics of the data in each task and the hyperparameters relating the tasks together. The article, as such, provides first insights on an offline control of multitask learning, which finds natural connections to the currently popular transfer learning paradigm.

Mots clés: Transfer Learning

Domaines: Informatique [cs]; Mathématiques [math]; Statistiques [stat]

Malik Tiomoko, Cosme Louart, Romain Couillet. Large Dimensional Asymptotics of Multi-Task Learning. ICASSP 2020 - IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE, May 2020, Barcelone (virtual), Spain. pp.8787-8791, ⟨10.1109/ICASSP40776.2020.9053557⟩. ⟨hal-02965810⟩