A Sparse Generative Model and its EM Algorithm for Variable Selection in High-Dimensional Regression

Abstract : We address the problem of Bayesian variable selection for high-dimensional linear regression. We consider a generative model that uses a spike-and-slab like prior distribution obtained by multiplying a deterministic binary vector, which traduces the sparsity of the problem, with a random Gaussian parameter vector. Such a model allows an expectation-maximization algorithm, optimizing a type-II log-likelihood, to be derived. This marginal log-likelihood involves an Occam's razor term, automatically penalizing the complexity, which is used for model selection. Albeit NP-hard, the algorithm we propose can be relaxed in order to infer a family of models. Model selection is eventually performed afterwards based on Occam's razor. We report numerical comparisons between our method, called spinyReg, and the most recent variable selection algorithms, including lasso, adaptive lasso and stability selection. SpinyReg turns out to perform well compared to those algorithms, especially regarding false detection rates.
Document type :
Preprints, Working Papers, ...
Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-01003395
Contributor : Charles Bouveyron <>
Submitted on : Wednesday, September 10, 2014 - 7:00:07 AM
Last modification on : Sunday, January 19, 2020 - 6:38:32 PM
Long-term archiving on: Thursday, December 11, 2014 - 11:40:22 AM

File

SpinyReg.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01003395, version 1

Citation

Charles Bouveyron, Julien Chiquet, Pierre Latouche, Pierre-Alexandre Mattei. A Sparse Generative Model and its EM Algorithm for Variable Selection in High-Dimensional Regression. 2014. ⟨hal-01003395v1⟩

Share

Metrics

Record views

547

Files downloads

240