Sparse Accelerated Exponential Weights

Pierre Gaillard 1 Olivier Wintenberger 2, 3
1 SIERRA - Statistical Machine Learning and Parsimony
DI-ENS - Département d'informatique de l'École normale supérieure, CNRS - Centre National de la Recherche Scientifique, Inria de Paris
Abstract : We consider the stochastic optimization problem where a convex function is minimized observing recursively the gradients. We introduce SAEW, a new procedure that accelerates exponential weights procedures with the slow rate 1/ √ T to procedures achieving the fast rate 1/T. Under the strong convexity of the risk, we achieve the optimal rate of convergence for approximating sparse parameters in R^d. The acceleration is achieved by using successive averaging steps in an online fashion. The procedure also produces sparse estimators thanks to additional hard threshold steps.
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01376808
Contributor : Pierre Gaillard <>
Submitted on : Monday, October 17, 2016 - 10:32:47 AM
Last modification on : Thursday, March 21, 2019 - 2:29:48 PM
Document(s) archivé(s) le : Friday, February 3, 2017 - 4:16:54 PM

Files

GaillardWintenberger2016-SAEW....
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01376808, version 1

Citation

Pierre Gaillard, Olivier Wintenberger. Sparse Accelerated Exponential Weights. 20th International Conference on Artificial Intelligence and Statistics (AISTATS), Apr 2017, Fort Lauderdale, United States. ⟨hal-01376808⟩

Share

Metrics

Record views

310

Files downloads

83