Self-normalization techniques for streaming confident regression

Odalric-Ambrym Maillard 1, *
* Corresponding author
1 TAO - Machine Learning and Optimisation
CNRS - Centre National de la Recherche Scientifique : UMR8623, Inria Saclay - Ile de France, UP11 - Université Paris-Sud - Paris 11, LRI - Laboratoire de Recherche en Informatique
Abstract : We consider, in a generic streaming regression setting, the problem of building a confidence interval (and distribution) on the next observation based on past observed data. The observations given to the learner are of the form (x, y) with y = f (x) + ξ, where x can have arbitrary dependency on the past observations, f is unknown and the noise ξ is sub-Gaussian conditionally on the past observations. Further, the observations are assumed to come from some external filtering process making the number of observations itself a random stopping time. In this challenging scenario that captures a large class of processes with non-anticipative dependencies, we study the ordinary, ridge, and kernel least-squares estimates and provide confidence intervals based on self-normalized vector-valued martingale techniques, applied to the estimation of the mean and of the variance. We then discuss how these adaptive confidence intervals can be used in order to detect a possible model mismatch as well as to estimate the future (self-information, quadratic, or transportation) loss of the learner at a next step.
Document type :
Preprints, Working Papers, ...
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01349727
Contributor : Odalric-Ambrym Maillard <>
Submitted on : Friday, July 29, 2016 - 10:07:36 AM
Last modification on : Thursday, April 5, 2018 - 12:30:12 PM
Document(s) archivé(s) le : Sunday, October 30, 2016 - 12:04:38 PM

File

HalSubmittedV1.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01349727, version 1

Citation

Odalric-Ambrym Maillard. Self-normalization techniques for streaming confident regression. 2016. ⟨hal-01349727v1⟩

Share

Metrics

Record views

190

Files downloads

130