The Dynamics of Learning: A Random Matrix Approach - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2018

The Dynamics of Learning: A Random Matrix Approach

Résumé

Understanding the learning dynamics of neural networks is one of the key issues for the improvement of optimization algorithms as well as for the theoretical comprehension of why deep neu-ral nets work so well today. In this paper, we introduce a random matrix-based framework to analyze the learning dynamics of a single-layer linear network on a binary classification problem, for data of simultaneously large dimension and size, trained by gradient descent. Our results provide rich insights into common questions in neural nets, such as overfitting, early stopping and the initialization of training, thereby opening the door for future studies of more elaborate structures and models appearing in today's neural networks.
Fichier principal
Vignette du fichier
liao18b.pdf (726.49 Ko) Télécharger le fichier
Origine : Fichiers éditeurs autorisés sur une archive ouverte
Loading...

Dates et versions

hal-01957704 , version 1 (17-12-2018)

Identifiants

  • HAL Id : hal-01957704 , version 1

Citer

Zhenyu Liao, Romain Couillet. The Dynamics of Learning: A Random Matrix Approach. ICML 2018 - 35th International Conference on Machine Learning, Jul 2018, Stockholm, Sweden. ⟨hal-01957704⟩
74 Consultations
32 Téléchargements

Partager

Gmail Facebook X LinkedIn More