H. Akaike, Information theory and an extension of the maximum likelihood principle, 2nd International Symposium on Information Theory, pp.267-281, 1973.

P. Alquier, Transductive and Inductive Adaptative Inference for Regression and Density Estimation, 2006.
URL : https://hal.archives-ouvertes.fr/tel-00119593

P. Alquier, PAC-Bayesian bounds for randomized empirical risk minimizers, Mathematical Methods of Statistics, vol.17, issue.4, pp.279-304, 2008.
DOI : 10.3103/S1066530708040017
URL : https://hal.archives-ouvertes.fr/hal-00354922

J. Audibert, Aggregated estimators and empirical complexity for least square regression. Annales de l'Institut Henri Poincaré: Probability and Statistics, pp.685-736, 2004.

J. Audibert, PAC-Bayesian Statistical Learning Theory, 2004.

F. Bach, Model-consistent sparse estimation through the bootstrap, 2009.
URL : https://hal.archives-ouvertes.fr/hal-00354771

A. Barron, A. Cohen, W. Dahmen, and R. Devore, Adaptative approximation and learning by greedy algorithms. The annals of statistics, pp.64-94, 2008.

P. J. Bickel, Y. Ritov, and A. Tsybakov, Simultaneous analysis of lasso and dantzig selector. The Annals of Statistics, pp.1705-1732, 2009.
URL : https://hal.archives-ouvertes.fr/hal-00401585

M. Bogdan, A. Chakrabarti, and J. K. Ghosh, Optimal rules for multiple testing and sparse multiple regression, p.3, 2008.

F. Bunea, A. Tsybakov, and M. Wegkamp, Sparsity oracle inequalities for the Lasso, Electronic Journal of Statistics, vol.1, issue.0, pp.169-194, 2007.
DOI : 10.1214/07-EJS008
URL : https://hal.archives-ouvertes.fr/hal-00160646

F. Bunea, A. B. Tsybakov, and M. H. Wegkamp, Aggregation for Gaussian regression, The Annals of Statistics, vol.35, issue.4, pp.1674-1697, 2007.
DOI : 10.1214/009053606000001587

T. Cai, G. Xu, and J. Zhang, On Recovery of Sparse Signals Via <formula formulatype="inline"> <tex Notation="TeX">$\ell _{1}$</tex></formula> Minimization, IEEE Transactions on Information Theory, vol.55, issue.7, pp.3388-3397, 2009.
DOI : 10.1109/TIT.2009.2021377

E. Candes and T. Tao, The dantzig selector: statistical estimation when p is much larger than n. The Annals of Statistics, 2007.

O. Catoni, A pac-bayesian approach to adaptative classification, Preprint Laboratoire de Probabilités et Modèles Aléatoires, 2003.

O. Catoni, Statistical Learning Theory and Stochastic Optimization, Lecture Notes in Mathematics, vol.1851, 2001.
DOI : 10.1007/b99352
URL : https://hal.archives-ouvertes.fr/hal-00104952

O. Catoni, PAC-Bayesian Supervised Classification (The Thermodynamics of Statistical Learning, Lecture Notes-Monograph Series. IMS, vol.56, 2007.
URL : https://hal.archives-ouvertes.fr/hal-00206119

S. S. Chen, D. L. Donoho, and M. A. Saunders, Atomic Decomposition by Basis Pursuit, SIAM Review, vol.43, issue.1, pp.129-159, 2001.
DOI : 10.1137/S003614450037906X

W. Cui and I. E. George, Empirical Bayes vs. fully Bayes variable selection, Journal of Statistical Planning and Inference, vol.138, issue.4, pp.888-900, 2008.
DOI : 10.1016/j.jspi.2007.02.011

A. Dalalyan and A. Tsybakov, Aggregation by exponential weighting, sharp oracle inequalities and sparsity, Machine Learning, pp.39-61, 2008.

A. S. Dalalyan and A. B. Tsybakov, Pac-bayesian bounds for the expected error of aggregation by exponential weights, 2009.

A. S. Dalalyan and A. B. Tsybakov, Mirror averaging with sparsity priors, Bernoulli, vol.18, issue.3, 2010.
DOI : 10.3150/11-BEJ361
URL : https://hal.archives-ouvertes.fr/hal-00461580

A. S. Dalalyan and A. B. Tsybakov, Sparse regression learning by aggregation and Langevin Monte-Carlo, Journal of Computer and System Sciences, vol.78, issue.5, 2010.
DOI : 10.1016/j.jcss.2011.12.023
URL : https://hal.archives-ouvertes.fr/hal-00362471

A. Dembo and O. Zeitouni, Large Deviation Techniques and Applications, 1998.

B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, Least angle regression, Ann. Statist, vol.32, issue.2, pp.407-499, 2004.

L. Frank and J. Friedman, A statistical view on some chemometrics regression tools, Technometrics, vol.16, pp.499-511, 1993.

I. E. George, The Variable Selection Problem, Journal of the American Statistical Association, vol.7, issue.2, pp.1304-1308, 2000.
DOI : 10.1214/aos/1176349027

I. E. George and R. E. Mcculloch, Approaches for bayesian model selection, Statistica Sinica, vol.7, pp.339-373, 1997.

S. Ghosh, Adaptive elastic net: an improvement of elastic net to achieve oracle properties, 2007.

P. J. Green, Reversible jump Markov chain Monte Carlo computation and Bayesian model determination, Biometrika, vol.82, issue.4, pp.711-732, 1995.
DOI : 10.1093/biomet/82.4.711

P. J. Green and S. Richardson, On bayesian analysis of mixtures with an unknown number of components, Journal of the Royal Statistical Society: series B (statistical methodology), vol.59, issue.4, pp.731-792, 1997.

C. Huang, G. L. Cheang, and A. Barron, Risk of penalized least squares, greedy selection and l1 penalization for flexible function libraries, 2008.

W. Jiang, Bayesian variable selection for high dimensionnal generalized linear models: Convergence rate of the fitted density. The Annals of Statistics, pp.1487-1511, 2007.

A. Juditsky, P. Rigollet, and A. Tsybakov, Learning by mirror averaging, The Annals of Statistics, vol.36, issue.5, pp.2183-2206, 2008.
DOI : 10.1214/07-AOS546
URL : https://hal.archives-ouvertes.fr/hal-00341026

V. Koltchinskii, Sparsity in empirical risk minimization. Annales de l'Institut Henri Poincaré, Probability and Statistics

E. L. Lehmann and G. Casella, Theory of Point Estimation, 1998.

G. Leung and A. R. Barron, Information Theory and Mixing Least-Squares Regressions, IEEE Transactions on Information Theory, vol.52, issue.8, pp.3396-3410, 2006.
DOI : 10.1109/TIT.2006.878172

F. Liang, R. Paulo, G. Molina, M. Clyde, and J. O. Berger, Priors for Bayesian Variable Selection, Journal of the American Statistical Association, vol.103, issue.481, pp.410-423, 2008.
DOI : 10.1198/016214507000001337

K. Lounici, Generalized mirror averaging and D-convex aggregation, Mathematical Methods of Statistics, vol.16, issue.3
DOI : 10.3103/S1066530707030040
URL : https://hal.archives-ouvertes.fr/hal-00204674

C. L. Mallows, Some comments on cp, Technometrics, vol.15, pp.661-676, 1973.

J. Marin and C. Robert, Bayesian Core: A practical approach to computational Bayesian analysis, 2007.

P. Massart, Concentration Inequalities and Model Selection (Saint-Flour Summer School on Probability Theory, 2003.

D. A. Mcallester, Some PAC-Bayesian theorems, Proceedings of the eleventh annual conference on Computational learning theory , COLT' 98, pp.230-234, 1998.
DOI : 10.1145/279943.279989

N. Meinshausen and P. Bühlmann, Stability selection, Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol.7, issue.4, 2010.
DOI : 10.1111/j.1467-9868.2010.00740.x

D. J. Nott and D. Leonte, Sampling Schemes for Bayesian Variable Selection in Generalized Linear Models, Journal of Computational and Graphical Statistics, vol.13, issue.2, pp.362-382, 2004.
DOI : 10.1198/1061860043425

R. Development and C. Team, R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, 2008.

P. Rigollet and A. B. Tsybakov, Exponential Screening and optimal rates of sparse estimation, The Annals of Statistics, vol.39, issue.2, 2010.
DOI : 10.1214/10-AOS854
URL : https://hal.archives-ouvertes.fr/hal-00606059

G. Schwarz, Estimating the Dimension of a Model, The Annals of Statistics, vol.6, issue.2, pp.461-464, 1978.
DOI : 10.1214/aos/1176344136

J. G. Scott and J. O. Berger, Bayes and empirical-bayes multiplicity adjustement in the variable-selection problem. The Annals of Statistics, 2010.

J. Shawe-taylor and R. Williamson, A pac analysis of a bayes estimator, Proceedings of the Tenth Annual Conference on Computational Learning Theory, COLT'97, pp.2-9, 1997.

R. Tibshirani, Regression shrinkage and selection via the lasso, Journal of the Royal Statistical Society B, vol.58, issue.1, pp.267-288, 1996.

A. B. Tsybakov, Optimal Rates of Aggregation, Computationnal Learning theory and Kernel Machines (COLT), pp.303-313, 2003.
DOI : 10.1007/978-3-540-45167-9_23
URL : https://hal.archives-ouvertes.fr/hal-00104867

S. A. Van-de-geer and P. Bühlmann, On the conditions used to prove oracle results for the Lasso, Electronic Journal of Statistics, vol.3, issue.0, pp.1360-1392, 2009.
DOI : 10.1214/09-EJS506

M. West, Bayesian factors in the "large p, small n" paradigm, Bayesian statistics, vol.7, pp.723-732, 2003.

Y. Yang, Aggregating regression procedures to improve performance, Bernoulli, vol.10, issue.1, pp.25-47, 2004.
DOI : 10.3150/bj/1077544602

T. Zhang, From epsilon-entropy to kl-entropy: analysis of minimum information complexity density estimation. The Annals of Statistics, pp.2180-2210, 2006.

H. Zou, The Adaptive Lasso and Its Oracle Properties, Journal of the American Statistical Association, vol.101, issue.476, pp.1418-1429, 2006.
DOI : 10.1198/016214506000000735

H. Zou and T. Hastie, Regularization and variable selection via the elastic net, Journal of the Royal Statistical Society: Series B (Statistical Methodology), vol.5, issue.2, pp.301-320, 2005.
DOI : 10.1073/pnas.201162998