Sparse Bayesian binary logistic regression using the split-and-augmented Gibbs sampler

Abstract : Logistic regression has been extensively used to perform classification in machine learning and signal/image processing. Bayesian formulations of this model with sparsity-inducing priors are particularly relevant when one is interested in drawing credibility intervals with few active coefficients. Along these lines, the derivation of efficient simulation-based methods is still an active research area because of the analytically challenging form of the binomial likelihood. This paper tackles the sparse Bayesian binary logistic regression problem by relying on the recent split-and-augmented Gibbs sampler (SPA). Contrary to usual data augmentation strategies, this Markov chain Monte Carlo (MCMC) algorithm scales in high dimension and divides the initial sampling problem into simpler ones. These sampling steps are then addressed with efficient state-of-the-art methods, namely proximal MCMC algorithms that can benefit from the recent closed-form expression of the proximal operator of the logistic cost function. SPA appears to be faster than efficient proximal MCMC algorithms and presents a reasonable computational cost compared to optimization-based methods with the advantage of producing credibility intervals. Experiments on handwritten digits classification problems illustrate the performances of the proposed approach.
Keywords : Gibbs sampler SPA
Document type :
Conference papers
Complete list of metadatas

Cited literature [31 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-02279425
Contributor : Open Archive Toulouse Archive Ouverte (oatao) <>
Submitted on : Thursday, September 5, 2019 - 11:47:34 AM
Last modification on : Thursday, October 24, 2019 - 2:44:13 PM

File

vono_22521.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02279425, version 1
  • OATAO : 22521

Citation

Maxime Vono, Nicolas Dobigeon, Pierre Chainais. Sparse Bayesian binary logistic regression using the split-and-augmented Gibbs sampler. IEEE International Workshop on Machine Learning for Signal Processing (MLSP 2018), Sep 2018, Aalborg, Denmark. pp.1-6. ⟨hal-02279425⟩

Share

Metrics

Record views

11

Files downloads

46