Skip to Main content Skip to Navigation
Journal articles

Uniform regret bounds over $R^d$ for the sequential linear regression problem with the square loss

Pierre Gaillard 1 Sébastien Gerchinovitz 2 Malo Huard 3, 4 Gilles Stoltz 3, 4
1 SIERRA - Statistical Machine Learning and Parsimony
DI-ENS - Département d'informatique de l'École normale supérieure, CNRS - Centre National de la Recherche Scientifique, Inria de Paris
4 CELESTE - Statistique mathématique et apprentissage
LMO - Laboratoire de Mathématiques d'Orsay, Inria Saclay - Ile de France
Abstract : We consider the setting of online linear regression for arbitrary deterministic sequences, with the square loss. We are interested in the aim set by Bartlett et al. (2015): obtain regret bounds that hold uniformly over all competitor vectors. When the feature sequence is known at the beginning of the game, they provided closed-form regret bounds of 2d B^2 ln T + O(1), where T is the number of rounds and B is a bound on the observations. Instead, we derive bounds with an optimal constant of 1 in front of the d B^2 ln T term. In the case of sequentially revealed features, we also derive an asymptotic regret bound of d B^2 ln T for any individual sequence of features and bounded observations. All our algorithms are variants of the online non-linear ridge regression forecaster, either with a data-dependent regularization or with almost no regularization.
Complete list of metadata

Cited literature [14 references]  Display  Hide  Download
Contributor : Gilles Stoltz <>
Submitted on : Tuesday, February 19, 2019 - 7:12:41 PM
Last modification on : Wednesday, June 9, 2021 - 10:00:10 AM
Long-term archiving on: : Monday, May 20, 2019 - 5:47:07 PM


Files produced by the author(s)


  • HAL Id : hal-01802004, version 2
  • ARXIV : 1805.11386


Pierre Gaillard, Sébastien Gerchinovitz, Malo Huard, Gilles Stoltz. Uniform regret bounds over $R^d$ for the sequential linear regression problem with the square loss. Proceedings of Machine Learning Research, PMLR, 2019, Proceedings of the 30th International Conference on Algorithmic Learning Theory, 98, pp.404-432. ⟨hal-01802004v2⟩



Record views


Files downloads