PAC-Bayesian Bounds for Sparse Regression Estimation with Exponential Weights - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2010

PAC-Bayesian Bounds for Sparse Regression Estimation with Exponential Weights

Résumé

We consider the sparse regression model where the number of parameters $p$ is larger than the sample size $n$. The difficulty when considering high-dimensional problems is to propose estimators achieving a good compromise between statistical and computational performances. The BIC estimator for instance performs well from the statistical point of view \cite{BTW07} but can be computed for values of $p$ of at most a few tens. The Lasso estimator is solution of a convex minimization problem. Hence it can be computed for large value of $p$. However stringent conditions on the design are required to establish the statistical properties of this estimator. Dalalyan and Tsybakov \cite{arnak} propose a method achieving a good compromise between the statistical and computational aspects of the problem. Their estimator can be computed for reasonably large $p$ and satisfies nice statistical properties under weak assumptions on the design. However, \cite{arnak} concerns only the empirical risk and proposes only results in expectation. In this paper, we propose an aggregation procedure similar to that of \cite{arnak} but with improved statistical performances. Our main result concerns the expected risk and is given in probability. We also propose a MCMC method to compute our estimator for reasonably large values of $p$.
Fichier principal
Vignette du fichier
RJMCMC.pdf (566.48 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-00465801 , version 1 (25-03-2010)
hal-00465801 , version 2 (09-04-2010)
hal-00465801 , version 3 (14-09-2010)

Identifiants

  • HAL Id : hal-00465801 , version 3

Citer

Pierre Alquier, Karim Lounici. PAC-Bayesian Bounds for Sparse Regression Estimation with Exponential Weights. 2010. ⟨hal-00465801v3⟩
413 Consultations
349 Téléchargements

Partager

Gmail Facebook X LinkedIn More