Logarithmic Regret for parameter-free Online Logistic Regression

Joseph de Vilmarest (1) , Olivier Wintenberger (1)

1 LPSM (UMR_8001) - Laboratoire de Probabilités, Statistique et Modélisation

We consider online optimization procedures in the context of logistic regression, focusing on the Extended Kalman Filter (EKF). We introduce a second-order algorithm close to the EKF, named Semi-Online Step (SOS), for which we prove a O(log(n)) regret in the adversarial setting, paving the way to similar results for the EKF. This regret bound on SOS is the first for such parameter-free algorithm in the adversarial logistic regression. We prove for the EKF in constant dynamics a O(log(n)) regret in expectation and in the well-specified logistic regression model.

Mots clés: Kalman filter; Logistic Regression; Online Optimization

Domaines: Apprentissage [cs.LG]; Statistiques [math.ST]; Intelligence artificielle [cs.AI]

Joseph de Vilmarest, Olivier Wintenberger. Logarithmic Regret for parameter-free Online Logistic Regression. 2019. ⟨hal-02046843⟩