HAL: inria-00552267, version 2

arXiv: 1101.1057

See detailed view

BibTeX,EndNote,... BibTeX - EndNote - TEI - RefWorks -

Available versions

v1 (2011-01-05)

v2 (2012-03-18)

v3 (2013-04-12)

Sparsity regret bounds for individual sequences in online linear regression

Sébastien Gerchinovitz 1, 2

(2011-01-05)

We consider the problem of online linear regression on arbitrary deterministic sequences when the ambient dimension d can be much larger than the number of time rounds T. We introduce the notion of sparsity regret bound, which is a deterministic online counterpart of recent risk bounds derived in the stochastic setting under a sparsity scenario. We prove such regret bounds for an online-learning algorithm called SeqSEW and based on exponential weighting and data-driven truncation. In a second part we apply a parameter-free version of this algorithm to the stochastic setting (regression model with random design). This yields risk bounds of the same flavor as in Dalalyan and Tsybakov (2011) but which solve two questions left open therein. In particular our risk bounds are adaptive (up to a logarithmic factor) to the unknown variance of the noise if the latter is Gaussian. We also address the regression model with fixed design.

1:	Département de Mathématiques et Applications (DMA)
	CNRS : UMR8553 – Ecole normale supérieure de Paris - ENS Paris
2:	CLASSIC (INRIA Paris - Rocquencourt)
	Ecole normale supérieure de Paris - ENS Paris – INRIA

Domain

:

Statistics/Machine Learning Statistics/Other Statistics Mathematics/Statistics Statistics/Statistics Theory Computer Science/Learning

Keywords: sparsity – online linear regression – individual sequences – adaptive regret bounds

inria-00552267, version 2

http://hal.inria.fr/inria-00552267

oai:hal.inria.fr:inria-00552267

From: Sébastien Gerchinovitz <>

Submitted on: Saturday, 17 March 2012 23:12:06

Updated on: Sunday, 18 March 2012 07:53:00