HAL will be down for maintenance from Friday, June 10 at 4pm through Monday, June 13 at 9am. More information
Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Self-normalization techniques for streaming confident regression

Odalric-Ambrym Maillard 1, *
* Corresponding author
1 TAO - Machine Learning and Optimisation
CNRS - Centre National de la Recherche Scientifique : UMR8623, Inria Saclay - Ile de France, UP11 - Université Paris-Sud - Paris 11, LRI - Laboratoire de Recherche en Informatique
Abstract : We consider, in a generic streaming regression setting, the problem of building a confidence interval (and distribution) on the next observation based on past observed data. The observations given to the learner are of the form (x, y) with y = f (x) + ξ, where x can have arbitrary dependency on the past observations, f is unknown and the noise ξ is sub-Gaussian conditionally on the past observations. Further, the observations are assumed to come from some external filtering process making the number of observations itself a random stopping time. In this challenging scenario that captures a large class of processes with non-anticipative dependencies, we study the ordinary, ridge, and kernel least-squares estimates and provide confidence intervals based on self-normalized vector-valued martingale techniques, applied to the estimation of the mean and of the variance. We then discuss how these adaptive confidence intervals can be used in order to detect a possible model mismatch as well as to estimate the future (self-information, quadratic, or transportation) loss of the learner at a next step.
Document type :
Preprints, Working Papers, ...
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-01349727
Contributor : Odalric-Ambrym Maillard Connect in order to contact the contributor
Submitted on : Friday, July 29, 2016 - 10:07:36 AM
Last modification on : Wednesday, September 16, 2020 - 5:11:14 PM
Long-term archiving on: : Sunday, October 30, 2016 - 12:04:38 PM

File

HalSubmittedV1.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01349727, version 1

Citation

Odalric-Ambrym Maillard. Self-normalization techniques for streaming confident regression. 2016. ⟨hal-01349727v1⟩

Share

Metrics

Record views

590

Files downloads

708