Relative entropy minimizing noisy non-linear neural network to approximate stochastic processes

Mathieu N Galtier (1) , Camille Marini (2) , Gilles Wainrib (3) , H. Jaeger (1)

1 JU-SES - School of Engineering and Science [Bremen]
2 O.I.E. - Centre Observation, Impacts, Énergie
3 LAGA - Laboratoire Analyse, Géométrie et Applications

A method is provided for designing and training noise-driven recurrent neural networks as models of stochastic processes. The method unifies and generalizes two known separate modeling approaches, Echo State Networks (ESN) and Linear Inverse Modeling (LIM), under the common principle of relative entropy minimization. The power of the new method is demonstrated on a stochastic approximation of the El Niño phenomenon studied in climate research.

Mots clés: Echo state networks; Linear inverse modeling; Relative entropy; Stochastic processes; El Nino southern oscillation

Domaines: Machine Learning [stat.ML]

Mathieu N Galtier, Camille Marini, Gilles Wainrib, H. Jaeger. Relative entropy minimizing noisy non-linear neural network to approximate stochastic processes. Neural Networks, 2014, 56, http://www.sciencedirect.com/science/article/pii/S0893608014000860. ⟨10.1016/j.neunet.2014.04.002⟩. ⟨hal-00994652⟩