PAC-Bayesian aggregation of affine estimators

Lucie Montuelle (1) , Erwan Le Pennec (2, 3)

1 LPMA - Laboratoire de Probabilités et Modèles Aléatoires
2 CMAP - Centre de Mathématiques Appliquées - Ecole Polytechnique
3 XPOP - Modélisation en pharmacologie de population

Aggregating estimators using exponential weights depending on their risk performs well in expectation, but sadly not in probability. Considering exponential weights of a penalized risk is a way to overcome this issue. We focus on the fixed design regression framework with sub-Gaussian noise and provide penalties allowing to obtain oracle inequalities in deviation for the aggregation of affine estimators. Sharp oracle inequalities are provided by a condition using the regression function's norm. MSC 2010 subject classifications: Primary 62G08; secondary 62J02.

Mots clés: Exponentially weighted aggregation; Regression; Oracle inequality; Deviation

Domaines: Statistiques [math.ST]; Théorie [stat.TH]

Lucie Montuelle, Erwan Le Pennec. PAC-Bayesian aggregation of affine estimators. 2016. ⟨hal-01070805v2⟩