Skip to Main content Skip to Navigation
Conference papers

Optimisation de la Topologie pour les Réseaux de Neurones Profonds

Ludovic Arnold 1, 2 Hélène Paugam-Moisy 3, 2 Michèle Sebag 1, 2
2 TAO - Machine Learning and Optimisation
CNRS - Centre National de la Recherche Scientifique : UMR8623, Inria Saclay - Ile de France, UP11 - Université Paris-Sud - Paris 11, LRI - Laboratoire de Recherche en Informatique
3 DM2L - Data Mining and Machine Learning
LIRIS - Laboratoire d'InfoRmatique en Image et Systèmes d'information
Abstract : Recently, deep neural networks have proven their ability to achieve excellent results on tasks such as classification and dimensionality reduction. The issue of hyper-parameter selection is decisive for these networks since the size of the search space increases exponentially with the number of layers. As a result, the grid-search approach is inappropriate and it is often left to the experimenter to ``guess'' sensible values for the hyper-parameters. In this study, we propose to select hyper-parameters layer after layer, on the basis of an unsupervised criterion, thus reducing to linear the complexity of the hyper-parameter selection procedure. Two unsupervised criteria are considered and the study focuses on determining an optimal number of neurons per layer. Experimentally, we show that the reconstruction error constitutes an adequate criterion for the layer-wise optimization of the number of neurons. In addition, we observe that the optimal size of layers tends to decrease when the number of training samples increases and we discuss this counter-intuitive result.
Complete list of metadata

Cited literature [20 references]  Display  Hide  Download
Contributor : Ludovic Arnold <>
Submitted on : Tuesday, December 1, 2009 - 1:34:03 PM
Last modification on : Thursday, June 17, 2021 - 3:46:12 AM
Long-term archiving on: : Tuesday, October 16, 2012 - 3:06:41 PM


Files produced by the author(s)


  • HAL Id : hal-00437538, version 1


Ludovic Arnold, Hélène Paugam-Moisy, Michèle Sebag. Optimisation de la Topologie pour les Réseaux de Neurones Profonds. 17e congrès francophone AFRIF-AFIA Reconnaissance des Formes et Intelligence Artificielle - RFIA 2010, Jan 2010, Caen, France. ⟨hal-00437538⟩



Record views


Files downloads