Statistical Methods for Neural Network Prediction Models

Guy-Michel Cloarec (1)

1 Technocentre Renault [Guyancourt]

Intelligent modeling techniques have evolved from the application field, where prior knowledge and common sense on the system behavior was implemented using linguistics or life mimetic systems (mainly Fuzzy and Neural network networks) where topology and parameters were determined by hook or by crook (also called educated guess where the intelligence in the person performing the modeling rather than in the techniques), to more rigorous modeling approaches where theoretical issues are addressed using statistical inference and where the uncertainties about the modeling are handled. Recent works tends to bring together the two approaches that seemed to occupy opposite extremes of the system modeling approaches. As a matter of fact, the so-called intelligent methods and the classical mathematical systems methods (or regression methods) are connected by a common denominator : The statistical theory field. Modeling methodologies are using more and more features of what is defined as multivariable statistical analysis field. In this study, we will examine how the neural network modeling field can benefit from the techniques of the statistical theory field. With an emphasis on time series modeling, we will review the various techniques used in the neural network design and propose a general modeling methodology. The key elements will be the analysis and preprocessing of the data set, the use of statistical methods to reconsider the neural networks as probabilistic models and for obtaining good performance estimates, how to use these for in statistical decision making methodology to determine the optimal topology, how to use statistical methods to improve parameter optimization and build a neural network structure that makes tractable multistep ahead prediction.

Mots clés: Neural Networks; Statistical Theory; Topology Optimization; Generalization; Bootstrap; Course of Dimensionality; Learning Algorithm Convergence; Trajectory Learning; Principal Component Analysis; Feature Extraction; Network Committees

Domaines: Automatique / Robotique; Méthodologie [stat.ME]; Réseau de neurones [cs.NE]; Traitement du signal et de l'image [eess.SP]; Systèmes et contrôle [cs.SY]; Théorie de l'information et codage [math.IT]; Optimisation et contrôle [math.OC]; Statistiques [math.ST]; Machine Learning [stat.ML]; Intelligence artificielle [cs.AI]; Automatique; Ingénierie, finance et science [cs.CE]; Calcul parallèle, distribué et partagé [cs.DC]; Algorithme et structure de données [cs.DS]; Ingénierie assistée par ordinateur; Modélisation et simulation

Guy-Michel Cloarec. Statistical Methods for Neural Network Prediction Models. [Research Report] Control Systems Group School of Electronic Engineering. Dublin City University. 1997. ⟨hal-03235964⟩