A Sparse Generative Model and its EM Algorithm for Variable Selection in High-Dimensional Regression - Archive ouverte HAL Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2014

A Sparse Generative Model and its EM Algorithm for Variable Selection in High-Dimensional Regression

Résumé

We address the problem of Bayesian variable selection for high-dimensional linear regression. We consider a generative model that uses a spike-and-slab like prior distribution obtained by multiplying a deterministic binary vector, which traduces the sparsity of the problem, with a random Gaussian parameter vector. Such a model allows an expectation-maximization algorithm, optimizing a type-II log-likelihood, to be derived. This marginal log-likelihood involves an Occam's razor term, automatically penalizing the complexity, which is used for model selection. Albeit NP-hard, the algorithm we propose can be relaxed in order to infer a family of models. Model selection is eventually performed afterwards based on Occam's razor. We report numerical comparisons between our method, called spinyReg, and the most recent variable selection algorithms, including lasso, adaptive lasso and stability selection. SpinyReg turns out to perform well compared to those algorithms, especially regarding false detection rates.
Fichier principal
Vignette du fichier
SpinyReg.pdf (1.66 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-01003395 , version 1 (10-09-2014)
hal-01003395 , version 2 (29-01-2015)

Identifiants

  • HAL Id : hal-01003395 , version 1

Citer

Charles Bouveyron, Julien Chiquet, Pierre Latouche, Pierre-Alexandre Mattei. A Sparse Generative Model and its EM Algorithm for Variable Selection in High-Dimensional Regression. 2014. ⟨hal-01003395v1⟩
934 Consultations
1012 Téléchargements

Partager

Gmail Facebook X LinkedIn More