On optimality of empirical risk minimization in linear aggregation

Adrien Saumard (1)

1 CREST - Centre de Recherche en Economie et Statistique [Bruz]

In the first part of this paper, we show that the small-ball condition, recently introduced by Mendelson (2015), may behave poorly for important classes of localized functions such as wavelets, leading to suboptimal estimates of the rate of convergence of ERM for the linear aggregation problem. In a second part, we derive optimal upper and lower bounds for the excess risk of ERM when the dictionary is made of trigonometric functions. While the validity of the small-ball condition remains essentially open in the Fourier case, we show strong connection between our results and concentration inequalities recently obtained for the excess risk in Chatterjee (2014) and van de Geer and Wainwright (2016).

Mots clés: empirical process theory; small-ball property; concentration inequality; empirical risk minimization; linear aggregation

Domaines: Théorie [stat.TH]

Adrien Saumard. On optimality of empirical risk minimization in linear aggregation. 2016. ⟨hal-01314571v2⟩