Self-Concordant Analysis of Generalized Linear Bandits with Forgetting

Yoan Russac (1, 2, 3, 4) , Louis Faury (5, 6) , Olivier Cappé (1, 4) , Aurélien Garivier (7)

1 DI-ENS - Département d'informatique - ENS Paris
2 CNRS - Centre National de la Recherche Scientifique
3 PSL - Université Paris sciences et lettres
4 VALDA - Value from Data
5 Criteo AI Lab
6 Télécom Paris
7 UMPA-ENSL - Unité de Mathématiques Pures et Appliquées

Contextual sequential decision problems with categorical or numerical observations are ubiquitous and Generalized Linear Bandits (GLB) offer a solid theoretical framework to address them. In contrast to the case of linear bandits, existing algorithms for GLB have two drawbacks undermining their applicability. First, they rely on excessively pessimistic concentration bounds due to the non-linear nature of the model. Second, they require either non-convex projection steps or burn-in phases to enforce boundedness of the estimators. Both of these issues are worsened when considering non-stationary models, in which the GLB parameter may vary with time. In this work, we focus on self-concordant GLB (which include logistic and Poisson regression) with forgetting achieved either by the use of a sliding window or exponential weights. We propose a novel confidence-based algorithm for the maximum-likehood estimator with forgetting and analyze its perfomance in abruptly changing environments. These results as well as the accompanying numerical simulations highlight the potential of the proposed approach to address non-stationarity in GLB.

Domaines: Intelligence artificielle [cs.AI]

Yoan Russac, Louis Faury, Olivier Cappé, Aurélien Garivier. Self-Concordant Analysis of Generalized Linear Bandits with Forgetting. AISTATS 2021 - International Conference on Artificial Intelligence and Statistics, Apr 2021, San Diego / Virtual, United States. ⟨hal-02984117v2⟩