Empirical risk minimization is optimal for the convex aggregation problem.

Guillaume Lecué (1)

1 LAMA - Laboratoire d'Analyse et de Mathématiques Appliquées

Let $F$ be a finite model of cardinality $M$ and denote by $\conv(F)$ its convex hull. The problem of convex aggregation is to construct a procedure having a risk as close as possible to the minimal risk over $\conv(F)$. Consider the bounded regression model with respect to the squared risk denoted by $R(\cdot)$. If $\ERMC$ denotes the empirical risk minimization procedure over $\conv(F)$ then we prove that for any $x>0$, with probability greater than $1-4\exp(-x)$, \begin{equation*} R(\ERMC)\leq \min_{f\in\conv(F)}R(f)+c_0\max\Big(\psi_n^{(C)}(M),\frac{x}{n}\Big) \end{equation*}where $c_0>0$ is an absolute constant and $\psi_n^{(C)}(M)$ is the optimal rate of convex aggregation defined in \cite{TsyCOLT07} by $\psinM=M/n$ when $M\leq \sqrt{n}$ and $\psinM=\sqrt{\log\big(eM/\sqrt{n}\big)/n}$ when $M>\sqrt{n}$.

Domaines: Théorie [stat.TH]

Guillaume Lecué. Empirical risk minimization is optimal for the convex aggregation problem.. 2011. ⟨hal-00736203⟩