General bound of overfitting for MLP regression models.

Abstract : Multilayer perceptrons (MLP) with one hidden layer have been used for a long time to deal with non-linear regression. However, in some task, MLP's are too powerful models and a small mean square error (MSE) may be more due to overfitting than to actual modelling. If the noise of the regression model is Gaussian, the overfitting of the model is totally determined by the behavior of the likelihood ratio test statistic (LRTS), however in numerous cases the assumption of normality of the noise is arbitrary if not false. In this paper, we present an universal bound for the overfitting of such model under weak assumptions, this bound is valid without Gaussian or identifiability assumptions. The main application of this bound is to give a hint about determining the true architecture of the MLP model when the number of data goes to infinite. As an illustration, we use this theoretical result to propose and compare effective criteria to find the true architecture of an MLP.
Type de document :
Article dans une revue
Neurocomputing, Elsevier, 2012, 90, pp.106-110. 〈10.1016/j.neucom.2011.11.028〉
Liste complète des métadonnées

Littérature citée [9 références]  Voir  Masquer  Télécharger

https://hal.archives-ouvertes.fr/hal-00655918
Contributeur : Joseph Rynkiewicz <>
Soumis le : mardi 3 janvier 2012 - 10:31:15
Dernière modification le : mercredi 9 mai 2012 - 09:28:36
Document(s) archivé(s) le : mercredi 4 avril 2012 - 02:25:49

Fichiers

revineghal.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Collections

Citation

Joseph Rynkiewicz. General bound of overfitting for MLP regression models.. Neurocomputing, Elsevier, 2012, 90, pp.106-110. 〈10.1016/j.neucom.2011.11.028〉. 〈hal-00655918〉

Partager

Métriques

Consultations de
la notice

175

Téléchargements du document

80