Combining a Relaxed EM Algorithm with Occam's Razor for Bayesian Variable Selection in High-Dimensional Regression

Abstract : We address the problem of Bayesian variable selection for high-dimensional lin-ear regression. We consider a generative model that uses a spike-and-slab-like prior distribution obtained by multiplying a deterministic binary vector, which traduces the sparsity of the problem, with a random Gaussian parameter vector. The origi-nality of the work is to consider inference through relaxing the model and using a type-II log-likelihood maximization based on an EM algorithm. Model selection is performed afterwards relying on Occam's razor and on a path of models found by the EM algorithm. Numerical comparisons between our method, called spinyReg, and state-of-the-art high-dimensional variable selection algorithms (such as lasso, adap-tive lasso, stability selection or spike-and-slab procedures) are reported. Competitive variable selection results and predictive performances are achieved on both simulated and real benchmark data sets. An original regression data set involving the predic-tion of the number of visitors of the Orsay museum in Paris using bike-sharing system data is also introduced, illustrating the efficiency of the proposed approach. An R package implementing the spinyReg method is currently under development and is available at https://r-forge.r-project.org/projects/spinyreg.
Complete list of metadatas

Cited literature [54 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01003395
Contributor : Charles Bouveyron <>
Submitted on : Thursday, January 29, 2015 - 11:08:41 AM
Last modification on : Friday, September 20, 2019 - 4:34:02 PM
Long-term archiving on: Wednesday, May 27, 2015 - 1:55:22 PM

File

spinyreg_jmva.pdf
Files produced by the author(s)

Licence


Distributed under a Creative Commons Attribution 4.0 International License

Identifiers

Citation

Pierre Latouche, Pierre-Alexandre Mattei, Charles Bouveyron, Julien Chiquet. Combining a Relaxed EM Algorithm with Occam's Razor for Bayesian Variable Selection in High-Dimensional Regression. Journal of Multivariate Analysis, Elsevier, 2015, ⟨10.1016/j.jmva.2015.09.004⟩. ⟨hal-01003395v2⟩

Share

Metrics

Record views

1303

Files downloads

1243