Optimisation de la Topologie pour les Réseaux de Neurones Profonds

Ludovic Arnold 1, 2 Hélène Paugam-Moisy 3, 2 Michèle Sebag 1, 2
2 TAO - Machine Learning and Optimisation
LRI - Laboratoire de Recherche en Informatique, UP11 - Université Paris-Sud - Paris 11, Inria Saclay - Ile de France, CNRS - Centre National de la Recherche Scientifique : UMR8623
3 DM2L - Data Mining and Machine Learning
LIRIS - Laboratoire d'InfoRmatique en Image et Systèmes d'information
Abstract : Recently, deep neural networks have proven their ability to achieve excellent results on tasks such as classification and dimensionality reduction. The issue of hyper-parameter selection is decisive for these networks since the size of the search space increases exponentially with the number of layers. As a result, the grid-search approach is inappropriate and it is often left to the experimenter to ``guess'' sensible values for the hyper-parameters. In this study, we propose to select hyper-parameters layer after layer, on the basis of an unsupervised criterion, thus reducing to linear the complexity of the hyper-parameter selection procedure. Two unsupervised criteria are considered and the study focuses on determining an optimal number of neurons per layer. Experimentally, we show that the reconstruction error constitutes an adequate criterion for the layer-wise optimization of the number of neurons. In addition, we observe that the optimal size of layers tends to decrease when the number of training samples increases and we discuss this counter-intuitive result.
Complete list of metadatas

Cited literature [20 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-00437538
Contributor : Ludovic Arnold <>
Submitted on : Tuesday, December 1, 2009 - 1:34:03 PM
Last modification on : Thursday, November 21, 2019 - 2:29:53 AM
Long-term archiving on: Tuesday, October 16, 2012 - 3:06:41 PM

File

otrnp2010.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-00437538, version 1

Citation

Ludovic Arnold, Hélène Paugam-Moisy, Michèle Sebag. Optimisation de la Topologie pour les Réseaux de Neurones Profonds. 17e congrès francophone AFRIF-AFIA Reconnaissance des Formes et Intelligence Artificielle - RFIA 2010, Jan 2010, Caen, France. ⟨hal-00437538⟩

Share

Metrics

Record views

895

Files downloads

1546