P. Alquier, J. Ridgway, and N. Chopin, On the properties of variational approximations of Gibbs posteriors, JMLR, vol.17, issue.239, pp.1-41, 2016.

A. Ambroladze, E. Parrado-hernández, and J. Shawe-taylor, Tighter PAC-Bayes bounds, NIPS, 2006.

A. Banerjee, On Bayesian bounds, Proceedings of the 23rd international conference on Machine learning , ICML '06, pp.81-88, 2006.
DOI : 10.1145/1143844.1143855

L. Bégin, P. Germain, F. Laviolette, and J. Roy, PAC-Bayesian theory for transductive learning, AISTATS, pp.105-113, 2014.

L. Bégin, P. Germain, F. Laviolette, and J. Roy, PAC-Bayesian bounds based on the Rényi divergence, AISTATS, pp.435-444, 2016.

P. G. Bissiri, C. C. Holmes, and S. G. Walker, A general framework for updating belief distributions, Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol.52, issue.5, 2016.
DOI : 10.1111/rssb.12158

S. Boucheron, G. Lugosi, and P. Massart, Concentration inequalities : a nonasymptotic theory of independence, 2013.
DOI : 10.1093/acprof:oso/9780199535255.001.0001

URL : https://hal.archives-ouvertes.fr/hal-00794821

O. Catoni, PAC-Bayesian supervised classification: the thermodynamics of statistical learning, Inst. of Mathematical Statistic, vol.56, 2007.
URL : https://hal.archives-ouvertes.fr/hal-00206119

S. Arnak, A. B. Dalalyan, and . Tsybakov, Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity, Machine Learning, pp.39-61, 2008.

Y. Freund, E. Robert, and . Schapire, A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting, Journal of Computer and System Sciences, vol.55, issue.1, pp.119-139, 1997.
DOI : 10.1006/jcss.1997.1504

P. Germain, A. Lacasse, F. Laviolette, and M. Marchand, PAC-Bayesian learning of linear classifiers, Proceedings of the 26th Annual International Conference on Machine Learning, ICML '09, pp.353-360, 2009.
DOI : 10.1145/1553374.1553419

P. Germain, A. Lacasse, F. Laviolette, M. Marchand, and J. Roy, Risk bounds for the majority vote: From a PAC-Bayesian analysis to a learning algorithm, JMLR, vol.16, 2015.

P. Germain, A. Habrard, F. Laviolette, and E. Morvant, A new PAC-Bayesian perspective on domain adaptation, ICML, pp.859-868, 2016.
URL : https://hal.archives-ouvertes.fr/hal-01307045

Z. Ghahramani, Probabilistic machine learning and artificial intelligence, Nature, vol.80, issue.7553, pp.452-459, 2015.
DOI : 10.1109/5.18626

P. Grünwald, The safe Bayesian -learning the learning rate via the mixability gap, ALT, 2012.

P. Grünwald and J. Langford, Suboptimal behavior of Bayes and MDL in classification under misspecification, Machine Learning, pp.119-149, 2007.

P. Grünwald and T. Van-ommen, Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing It, 1412.

D. Peter and . Grünwald, The Minimum Description Length Principle, 2007.

D. Peter, N. A. Grünwald, and . Mehta, Fast rates with unbounded losses. CoRR, abs, 1605.

I. Guyon, A. Saffari, G. Dror, and G. C. Cawley, Model selection: Beyond the Bayesian/frequentist divide, JMLR, vol.11, pp.61-87, 2010.

T. Hazan, S. Maji, J. Keshet, and T. S. Jaakkola, Learning efficient random maximum a-posteriori predictors with non-decomposable loss functions, NIPS, pp.1887-1895, 2013.

H. William, J. O. Jeffreys, and . Berger, Ockham's razor and Bayesian analysis, American Scientist, 1992.

M. Sham, A. Y. Kakade, and . Ng, Online bounds for Bayesian algorithms, NIPS, pp.641-648, 2004.

A. Lacoste, Agnostic Bayes, 2015.

S. Lacoste-julien, F. Huszar, and Z. Ghahramani, Approximate inference for the losscalibrated Bayesian, AISTATS, pp.416-424, 2011.

J. Langford and M. Seeger, Bounds for averaging classifiers, 2001.

J. Langford and J. Shawe-taylor, PAC-Bayes & margins, NIPS, pp.423-430, 2002.

G. Lever, F. Laviolette, and J. Shawe-taylor, Tighter PAC-Bayes bounds through distribution-dependent priors, Theoretical Computer Science, vol.473, pp.4-28, 2013.
DOI : 10.1016/j.tcs.2012.10.013

URL : http://dx.doi.org/10.1016/j.tcs.2012.10.013

J. C. David and . Mackay, Bayesian interpolation, Neural Computation, vol.4, issue.3, pp.415-447, 1992.

A. Maurer, A note on the PAC-Bayesian theorem. CoRR, cs, 2004.

D. Mcallester, Some PAC-Bayesian theorems, Proceedings of the eleventh annual conference on Computational learning theory , COLT' 98, pp.355-363, 1999.
DOI : 10.1145/279943.279989

D. Mcallester, PAC-Bayesian stochastic model selection, Machine Learning, pp.5-21, 2003.

D. Mcallester and J. Keshet, Generalization bounds and consistency for latent structural probit and ramp loss, NIPS, pp.2205-2212, 2011.

R. Meir and T. Zhang, Generalization error bounds for Bayesian mixture algorithms, Journal of Machine Learning Research, vol.4, pp.839-860, 2003.

Y. Andrew, M. I. Ng, and . Jordan, On discriminative vs. generative classifiers: A comparison of logistic regression and naive Bayes, NIPS, pp.841-848, 2001.

A. Noy and K. Crammer, Robust forward algorithms via PAC-Bayes and Laplace distributions, AISTATS, 2014.
DOI : 10.1007/978-3-319-21852-6_25

A. Pentina and C. H. Lampert, A PAC-Bayesian bound for lifelong learning, ICML, 2014.

C. Rasmussen and C. Williams, Gaussian Processes in Machine Learning, 2006.
DOI : 10.1162/089976602317250933

J. Rousseau, On the Frequentist Properties of Bayesian Nonparametric Methods, Annual Review of Statistics and Its Application, vol.3, issue.1, pp.211-231, 2016.
DOI : 10.1146/annurev-statistics-041715-033523

URL : https://hal.archives-ouvertes.fr/hal-01252919

A. F. George, A. J. Seber, and . Lee, Linear regression analysis, 2012.

M. Seeger, 10.1162/153244303765208377, CrossRef Listing of Deleted DOIs, vol.7, issue.5, pp.233-269, 2002.
DOI : 10.1016/S0004-3702(98)00002-2

M. Seeger, Bayesian Gaussian Process Models: PAC-Bayesian Generalisation Error Bounds and Sparse Approximations, 2003.
DOI : 10.1162/153244303765208386

URL : http://citeseerx.ist.psu.edu/viewdoc/summary?doi=

Y. Seldin and N. Tishby, PAC-Bayesian analysis of co-clustering and beyond, 2010.

Y. Seldin, P. Auer, F. Laviolette, J. Shawe-taylor, and R. Ortner, PAC-Bayesian analysis of contextual bandits, NIPS, pp.1683-1691, 2011.

Y. Seldin, F. Laviolette, N. Cesa-bianchi, J. Shawe-taylor, and P. Auer, PAC-Bayesian Inequalities for Martingales, UAI, 2012.
DOI : 10.1109/TIT.2012.2211334

URL : http://arxiv.org/abs/1110.6886

J. Shawe-taylor and R. C. Williamson, A PAC analysis of a Bayesian estimator, Proceedings of the tenth annual conference on Computational learning theory , COLT '97, 1997.
DOI : 10.1145/267460.267466

O. Ilya, Y. Tolstikhin, and . Seldin, PAC-Bayes-empirical-Bernstein inequality, NIPS, 2013.

T. Zhang, Information-theoretic upper and lower bounds for statistical estimation, IEEE Transactions on Information Theory, vol.52, issue.4, pp.1307-1321, 2006.
DOI : 10.1109/TIT.2005.864439