 |
|
|
|
| HAL : hal-00488338, version 1 |
| Fiche détaillée |  Récupérer au format |
|
|
| 19th European Conference on Artificial Intelligence (ECAI'10), Lisbon : Portugal (2010) |
|
|
|
|
| Unsupervised Layer-Wise Model Selection in Deep Neural Networks |
|
|
| Ludovic Arnold 1, 2Hélène Paugam-Moisy 2, 3 |
|
|
| (16/08/2010) |
|
|
|
| Deep Neural Networks (DNN) propose a new and efficient ML architecture based on the layer-wise building of several representation layers. A critical issue for DNNs remains model selection, e.g. selecting the number of neurons in each DNN layer. The hyper-parameter search space exponentially increases with the number of layers, making the popular grid search-based approach used for finding good hyper-parameter values intractable. The question investigated in this paper is whether the unsupervised, layer-wise methodology used to train a DNN can be extended to model selection as well. The proposed approach, considering an unsupervised criterion, empirically examines whether model selection is a modular optimization problem, and can be tackled in a layer-wise manner. Preliminary results on the MNIST data set suggest the answer is positive. Further, some unexpected results regarding the optimal size of layers depending on the training process, are reported and discussed. |
|
|
|
|
|
|
|
|
|
|
|
| 1 : | Laboratoire de Recherche en Informatique (LRI) |
|
|
|
CNRS : UMR8623 – Université Paris XI - Paris Sud |
|
|
| 2 : | TAO (INRIA Saclay - Ile de France) |
|
|
|
INRIA – CNRS : UMR8623 – Université Paris XI - Paris Sud |
|
|
| 3 : | Laboratoire d'InfoRmatique en Images et Systèmes d'Information (LIRIS) |
|
|
|
CNRS : UMR5205 – Université Claude Bernard - Lyon I – Université Lumière - Lyon II – Institut National des Sciences Appliquées (INSA) - Lyon – Ecole Centrale de Lyon |
|
|
|
|
|
|
|
|
|
| Domaine | : | Informatique/Réseau de neurones |
|
|
| Learning – Deep Neural Networks – Model Selection |
|
|
|
| Liste des fichiers attachés à ce document : | |||||
|
|||||
|
|
|
| hal-00488338, version 1 | |
| http://hal.archives-ouvertes.fr/hal-00488338 | |
| oai:hal.archives-ouvertes.fr:hal-00488338 | |
| Contributeur : Ludovic Arnold | |
| Soumis le : Mardi 1 Juin 2010, 17:34:08 | |
| Dernière modification le : Mardi 1 Juin 2010, 21:09:26 | |
