PAC-Bayesian risk bounds for group-analysis sparse regression by exponential weighting

Duy Tung Luu (1) , Jalal M. Fadili (1) , Christophe Chesneau (2)

1 Equipe Image - Laboratoire GREYC - UMR6072
2 LMNO - Laboratoire de Mathématiques Nicolas Oresme

In this paper, we consider a high-dimensional non-parametric regression model with fixed design and i.i.d. random errors. We propose a powerful estimator by exponential weighted aggregation (EWA) with a group-analysis sparsity promoting prior on the weights. We prove that our estimator satisfies a sharp group-analysis sparse oracle inequality with a small remainder term ensuring its good theoretical performances. We also propose a forward-backward proximal Langevin Monte-Carlo algorithm to sample from the target distribution (which is not smooth nor log-concave) and derive its guarantees. In turn, this allows us to implement our estimator and validate it on some numerical experiments.

Mots clés: High-dimensional regression; group-analysis sparse oracle inequality; frame; sparse oracle inequality; forward-backward Langevin Monte-Carlo; forward-backward Langevin Monte-Carlo algorithm; exponential weighted aggregation; sparse learning; sparsity; group-analysis sparsity; group-analysis

Domaines: Statistiques [math.ST]

Duy Tung Luu, Jalal M. Fadili, Christophe Chesneau. PAC-Bayesian risk bounds for group-analysis sparse regression by exponential weighting. Journal of Multivariate Analysis, In press. ⟨hal-01367742v4⟩