# Optimisation de la Topologie pour les Réseaux de Neurones Profonds

2 TAO - Machine Learning and Optimisation
CNRS - Centre National de la Recherche Scientifique : UMR8623, Inria Saclay - Ile de France, UP11 - Université Paris-Sud - Paris 11, LRI - Laboratoire de Recherche en Informatique
3 DM2L - Data Mining and Machine Learning
LIRIS - Laboratoire d'InfoRmatique en Image et Systèmes d'information
Abstract : Recently, deep neural networks have proven their ability to achieve excellent results on tasks such as classification and dimensionality reduction. The issue of hyper-parameter selection is decisive for these networks since the size of the search space increases exponentially with the number of layers. As a result, the grid-search approach is inappropriate and it is often left to the experimenter to guess'' sensible values for the hyper-parameters. In this study, we propose to select hyper-parameters layer after layer, on the basis of an unsupervised criterion, thus reducing to linear the complexity of the hyper-parameter selection procedure. Two unsupervised criteria are considered and the study focuses on determining an optimal number of neurons per layer. Experimentally, we show that the reconstruction error constitutes an adequate criterion for the layer-wise optimization of the number of neurons. In addition, we observe that the optimal size of layers tends to decrease when the number of training samples increases and we discuss this counter-intuitive result.
Keywords :
Document type :
Conference papers
Domain :

Cited literature [20 references]

https://hal.archives-ouvertes.fr/hal-00437538
Contributor : Ludovic Arnold <>
Submitted on : Tuesday, December 1, 2009 - 1:34:03 PM
Last modification on : Thursday, June 17, 2021 - 3:46:12 AM
Long-term archiving on: : Tuesday, October 16, 2012 - 3:06:41 PM

### File

otrnp2010.pdf
Files produced by the author(s)

### Identifiers

• HAL Id : hal-00437538, version 1

### Citation

Ludovic Arnold, Hélène Paugam-Moisy, Michèle Sebag. Optimisation de la Topologie pour les Réseaux de Neurones Profonds. 17e congrès francophone AFRIF-AFIA Reconnaissance des Formes et Intelligence Artificielle - RFIA 2010, Jan 2010, Caen, France. ⟨hal-00437538⟩

Record views