Sharp Oracle Inequalities for Low-complexity Priors

Duy Tung Luu (1) , Jalal M. Fadili (1) , Christophe Chesneau (2)

1 Equipe Image - Laboratoire GREYC - UMR6072
2 LMNO - Laboratoire de Mathématiques Nicolas Oresme

In this paper, we consider a high-dimensional statistical estimation problem in which the the number of parameters is comparable or larger than the sample size. We present a unified analysis of the performance guarantees of exponential weighted aggregation and penalized estimators with a general class of data losses and priors which encourage objects which conform to some notion of simplicity/complexity. More precisely, we show that these two estimators satisfy sharp oracle inequalities for prediction ensuring their good theoretical performances. We also highlight the differences between them. When the noise is random, we provide oracle inequalities in probability using concentration inequalities. These results are then applied to several instances including the Lasso, the group Lasso, their analysis-type counterparts, the $\ell_\infty$ and the nuclear norm penalties. All our estimators can be efficiently implemented using proximal splitting algorithms.

Mots clés: low-complexity models; exponential weighted aggregation; High-dimensional regression; penalized estimation; oracle inequality

Domaines: Statistiques [math.ST]

Duy Tung Luu, Jalal M. Fadili, Christophe Chesneau. Sharp Oracle Inequalities for Low-complexity Priors. Annals of the Institute of Statistical Mathematics, inPress. ⟨hal-01422476v5⟩