Harnessing neural networks: A random matrix approach

Cosme Louart 1 Romain Couillet 2
1 LVIC - Laboratoire Vision et Ingénierie des Contenus
DIASI - Département Intelligence Ambiante et Systèmes Interactifs : DRT/LIST/DIASI
Abstract : Abstract : This article proposes an original approach to the performance understanding of large dimensional neural networks. In this preliminary study, we study a single hidden layer feed-forward network with random input connections (also called extreme learning machine) which performs a simple regression task. By means of a new random matrix result, we prove that, as the size and cardinality of the input data and the number of neurons grow large, the network performance is asymptotically deterministic. This entails a better comprehension of the effects of the hyper-parameters (activation function, number of neurons, etc.) under this simple setting, thereby paving the path to the harnessing of more involved structures.
Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-01962073
Contributor : Cosme Louart <>
Submitted on : Thursday, December 20, 2018 - 1:20:44 PM
Last modification on : Wednesday, June 26, 2019 - 1:31:51 AM

Identifiers

Citation

Cosme Louart, Romain Couillet. Harnessing neural networks: A random matrix approach. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2017), Mar 2017, New Orleans, United States. ⟨10.1109/ICASSP.2017.7952563⟩. ⟨hal-01962073⟩

Share

Metrics

Record views

242