Recursive ridge regression using second-order stochastic algorithms

Antoine Godichon-Baggioni (1) , Wei Lu (2) , Bruno Portier (2)

1 LPSM (UMR_8001) - Laboratoire de Probabilités, Statistique et Modélisation
2 LMI - Laboratoire de Mathématiques de l'INSA de Rouen Normandie

Recursive second-order stochastic algorithms are presented for solving ridge regression problems in the linear and binary logistic case. The proposed algorithms allow us to update the estimates of ridge solution when the data arrive in continuous flow. We establish the almost sure convergence with rate of proposed algorithms. Numerical experiments on simulated and real-world data show the advantages of our algorithms compared to alternative methods.

Mots clés: Ridge regression; Stochastic optimization; Stochastic Newton algorithm; Recursive estimation; Machine learning

Domaines: Statistiques [math.ST]

Antoine Godichon-Baggioni, Wei Lu, Bruno Portier. Recursive ridge regression using second-order stochastic algorithms. 2022. ⟨hal-03858834v2⟩