# Non-stationary Online Regression

2 SIERRA - Statistical Machine Learning and Parsimony
DI-ENS - Département d'informatique de l'École normale supérieure, CNRS - Centre National de la Recherche Scientifique, Inria de Paris
3 Thoth - Apprentissage de modèles à partir de données massives
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann
Abstract : Online forecasting under a changing environment has been a problem of increasing importance in many real-world applications. In this paper, we consider the meta-algorithm presented in Zhang et al. 2017 combined with different subroutines. We show that an expected cumulative error of order $\tilde{O}(n^{1/3} C_n^{2/3})$ can be obtained for non-stationary online linear regression where the total variation of parameter sequence is bounded by $C_n$. Our paper extends the result of online forecasting of one-dimensional time-series as proposed in Baby et al. 2019 to general $d$-dimensional non-stationary linear regression. We improve the rate $O(\sqrt{n C_n})$ obtained by Zhang et al. 2017 and Besbes et al. 2015. We further extend our analysis to non-stationary online kernel regression. Similar to the non-stationary online regression case, we use the meta-procedure of Zhang et al. 2017 combined with Kernel-AWV (Jezequel et al. 2020) to achieve an expected cumulative controlled by the effective dimension of the RKHS and the total variation of the sequence. To the best of our knowledge, this work is the first extension of non-stationary online regression to non-stationary kernel regression. Finally, we empirically evaluate our method using several existing benchmarks and also compare it to the theoretical bounds obtained in this paper.
Keywords :
Document type :
Preprints, Working Papers, ...
Domain :

https://hal.archives-ouvertes.fr/hal-02998781
Contributor : Pierre Gaillard Connect in order to contact the contributor
Submitted on : Tuesday, November 10, 2020 - 4:27:34 PM
Last modification on : Friday, October 15, 2021 - 1:41:22 PM
Long-term archiving on: : Friday, February 12, 2021 - 12:21:07 PM

### Files

non-stationary-aistats.pdf
Files produced by the author(s)

### Identifiers

• HAL Id : hal-02998781, version 1
• ARXIV : 2011.06957

### Citation

Anant Raj, Pierre Gaillard, Christophe Saad. Non-stationary Online Regression. 2020. ⟨hal-02998781⟩

Record views