HAL : hal-00004908, version 2

arXiv : math.ST/0505333

Fiche détaillée

Récupérer au format BibTeX - EndNote - TEI - RefWorks -

Versions disponibles :

v1 (16-05-2005)

v2 (07-03-2006)

Recursive Aggregation of Estimators by Mirror Descent Algorithm with Averaging

Anatoli Juditsky 1, Alexander Nazin 2, Alexandre Tsybakov 3, Nicolas Vayatis 3

(12/05/2005)

We consider a recursive algorithm to construct an aggregated estimator from a finite number of base decision rules in the classification problem. The estimator approximately minimizes a convex risk functional under the l1-constraint. It is defined by a stochastic version of the mirror descent algorithm (i.e., of the method which performs gradient descent in the dual space) with an additional averaging. The main result of the paper is an upper bound for the expected accuracy of the proposed estimator. This bound is of the order $\sqrt{(\log M)/t}$ with an explicit and small constant factor, where $M$ is the dimension of the problem and $t$ stands for the sample size. A similar bound is proved for a more general setting that covers, in particular, the regression model with squared loss.

1 :	Laboratoire de Modélisation et Calcul (LMC - IMAG)
	CNRS : UMR5523 – Université Joseph Fourier - Grenoble I – Institut National Polytechnique de Grenoble (INPG)
2 :	Institute of Control Sciences (ICS)
	Institute of Control Sciences
3 :	Laboratoire de Probabilités et Modèles Aléatoires (LPMA)
	CNRS : UMR7599 – Université Paris VI - Pierre et Marie Curie – Université Paris VII - Paris Diderot

Domaine

:

Mathématiques/Statistiques

Mots Clés : aggregation – stochastic approximation – classification – svm – convex risk minimization – rates of convergence – online learning

hal-00004908, version 2

http://hal.archives-ouvertes.fr/hal-00004908

oai:hal.archives-ouvertes.fr:hal-00004908

Contributeur : Nicolas Vayatis <>

Soumis le : Mardi 7 Mars 2006, 19:12:43

Dernière modification le : Mardi 7 Mars 2006, 19:17:40