PAC-Bayesian Theory Meets Bayesian Inference

Abstract : We exhibit a strong link between frequentist PAC-Bayesian bounds and the Bayesian marginal likelihood. That is, for the negative log-likelihood loss function, we show that the minimization of PAC-Bayesian generalization bounds maximizes the Bayesian marginal likelihood. This provides an alternative explanation to the Bayesian Occam's razor criteria, under the assumption that the data is generated by a i.i.d. distribution. Moreover, as the negative log-likelihood is an unbounded loss function, we motivate and propose a PAC-Bayesian theorem tailored for the sub-Gamma loss family, and we show that our approach is sound on classical Bayesian linear regression tasks.
Document type :
Preprints, Working Papers, ...
Complete list of metadatas
Contributor : Pascal Germain <>
Submitted on : Tuesday, May 31, 2016 - 3:40:42 PM
Last modification on : Tuesday, April 24, 2018 - 5:20:15 PM
Long-term archiving on: Thursday, September 1, 2016 - 11:29:32 AM


Files produced by the author(s)


  • HAL Id : hal-01324072, version 1
  • ARXIV : 1605.08636


Pascal Germain, Francis Bach, Alexandre Lacoste, Simon Lacoste-Julien. PAC-Bayesian Theory Meets Bayesian Inference. 2016. ⟨hal-01324072v1⟩



Record views


Files downloads