Sharp Oracle Inequalities for Low-complexity Priors

Tung Duy Luu (1) , Jalal Fadili (1) , Christophe Chesneau (2)

1 Equipe Image - Laboratoire GREYC - UMR6072
2 LMNO - Laboratoire de Mathématiques Nicolas Oresme

In this paper, we consider a high-dimensional linear regression model with fixed design. We present a unified analysis of the performance guarantees of exponential weighted aggregation and penalized es-timators with a general class of priors which encourage objects which conform to some notion of sim-plicity/complexity. More precisely, we show that these two estimators satisfy sharp oracle inequalities for prediction ensuring their good theoretical performances. We also highlight the differences between them. The results are then applied to several instances including the Lasso, the group Lasso, their analysis-type counterparts, the ∞ and the nuclear norm penalties. When the noise is random, we provide oracle inequalities in probability under mild assumptions on the noise distribution. These estimators can be efficiently implemented using proximal splitting algorithms.

Mots clés: High-dimensional regression; exponential weighted aggregation; penalized estimation; oracle inequality; low-complexity models

Domaines: Statistiques [math.ST]

Tung Duy Luu, Jalal Fadili, Christophe Chesneau. Sharp Oracle Inequalities for Low-complexity Priors. 2017. ⟨hal-01422476v3⟩