An efficient stochastic Newton algorithm for parameter estimation in logistic regressions

Bernard Bercu (1) , Antoine Godichon (2) , Bruno Portier (2)

1 IMB - Institut de Mathématiques de Bordeaux
2 LMI - Laboratoire de Mathématiques de l'INSA de Rouen Normandie

Logistic regression is a well-known statistical model which is commonly used in the situation where the output is a binary random variable. It has a wide range of applications including machine learning, public health, social sciences, ecology and econometry. In order to estimate the unknown parameters of logistic regression with data streams arriving sequentially and at high speed, we focus our attention on a recursive stochastic algorithm. More precisely, we investigate the asymptotic behavior of a new stochastic Newton algorithm. It enables to easily update the estimates when the data arrive sequentially and to have research steps in all directions. We establish the almost sure convergence of our stochastic Newton algorithm as well as its asymptotic normality. All our theoretical results are illustrated by numerical experiments.

Domaines: Statistiques [stat]

Bernard Bercu, Antoine Godichon, Bruno Portier. An efficient stochastic Newton algorithm for parameter estimation in logistic regressions. SIAM Journal on Control and Optimization, 2020, 58 (1), pp.348-367. ⟨10.1137/19M1261717⟩. ⟨hal-02103041⟩