A Random Matrix and Concentration Inequalities framework for Neural Networks Analysis

Cosme Louart 1, 2 Romain Couillet 3
1 LVIC - Laboratoire Vision et Ingénierie des Contenus
DIASI - Département Intelligence Ambiante et Systèmes Interactifs : DRT/LIST/DIASI
2 GIPSA-CICS - CICS
GIPSA-DIS - Département Images et Signal
Abstract : This article provides a theoretical analysis of the asymptotic performance of a regression or classification task performed by a simple random neural network. This result is obtained by leveraging a new framework at the crossroads between random matrix theory and the concentration of measure theory. This approach is of utmost interest for neural network analysis at large in that it naturally dismisses the difficulty induced by the non-linear activation functions, so long that these are Lipschitz functions. As an application, we provide formulas for the limiting law of the random neural network output and compare them conclusively to those obtained practically on handwritten digits databases.
Complete list of metadatas

Cited literature [19 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01962077
Contributor : Cosme Louart <>
Submitted on : Thursday, January 10, 2019 - 1:15:47 PM
Last modification on : Friday, March 8, 2019 - 4:29:44 PM
Long-term archiving on : Thursday, April 11, 2019 - 2:00:28 PM

File

articleICASSP_2018.pdf
Files produced by the author(s)

Identifiers

Citation

Cosme Louart, Romain Couillet. A Random Matrix and Concentration Inequalities framework for Neural Networks Analysis. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2018), Apr 2018, Calgary, Canada. ⟨10.1109/ICASSP.2018.8462001⟩. ⟨hal-01962077⟩

Share

Metrics

Record views

238

Files downloads

161