Combining a Relaxed EM Algorithm with Occam's Razor for Bayesian Variable Selection in High-Dimensional Regression

Abstract : We address the problem of Bayesian variable selection for high-dimensional lin-ear regression. We consider a generative model that uses a spike-and-slab-like prior distribution obtained by multiplying a deterministic binary vector, which traduces the sparsity of the problem, with a random Gaussian parameter vector. The origi-nality of the work is to consider inference through relaxing the model and using a type-II log-likelihood maximization based on an EM algorithm. Model selection is performed afterwards relying on Occam's razor and on a path of models found by the EM algorithm. Numerical comparisons between our method, called spinyReg, and state-of-the-art high-dimensional variable selection algorithms (such as lasso, adap-tive lasso, stability selection or spike-and-slab procedures) are reported. Competitive variable selection results and predictive performances are achieved on both simulated and real benchmark data sets. An original regression data set involving the predic-tion of the number of visitors of the Orsay museum in Paris using bike-sharing system data is also introduced, illustrating the efficiency of the proposed approach. An R package implementing the spinyReg method is currently under development and is available at https://r-forge.r-project.org/projects/spinyreg.
Type de document :
Article dans une revue
Journal of Multivariate Analysis, Elsevier, 2015, <10.1016/j.jmva.2015.09.004>
Liste complète des métadonnées


https://hal.archives-ouvertes.fr/hal-01003395
Contributeur : Charles Bouveyron <>
Soumis le : jeudi 29 janvier 2015 - 11:08:41
Dernière modification le : vendredi 17 février 2017 - 16:14:20
Document(s) archivé(s) le : mercredi 27 mai 2015 - 13:55:22

Fichier

spinyreg_jmva.pdf
Fichiers produits par l'(les) auteur(s)

Licence


Distributed under a Creative Commons Paternité 4.0 International License

Identifiants

Citation

Pierre Latouche, Pierre-Alexandre Mattei, Charles Bouveyron, Julien Chiquet. Combining a Relaxed EM Algorithm with Occam's Razor for Bayesian Variable Selection in High-Dimensional Regression. Journal of Multivariate Analysis, Elsevier, 2015, <10.1016/j.jmva.2015.09.004>. <hal-01003395v2>

Partager

Métriques

Consultations de
la notice

759

Téléchargements du document

684