A Random Matrix Approach to Echo-State Neural Networks

Romain Couillet (1) , Gilles Wainrib (2) , Hafiz Tiomoko Ali (1) , Harry Sevi (3)

1 L2S - Laboratoire des signaux et systèmes
2 DI-ENS - Département d'informatique - ENS Paris
3 ENS de Lyon - École normale supérieure de Lyon

Recurrent neural networks, especially in their linear version, have provided many qualitative insights on their performance under different configurations. This article provides, through a novel random matrix framework, the quantitative counterpart of these performance results, particularly in the case of echo-state networks. Beyond mere insights, our approach conveys a deeper understanding on the core mechanism under play for both training and testing.

Domaines: Intelligence artificielle [cs.AI]; Réseau de neurones [cs.NE]; Apprentissage [cs.LG]; Statistiques [math.ST]

Romain Couillet, Gilles Wainrib, Hafiz Tiomoko Ali, Harry Sevi. A Random Matrix Approach to Echo-State Neural Networks. International Conference on Machine Learning (ICML 2016), Jun 2016, New York, United States. ⟨hal-01812026⟩