A general procedure to combine estimators

Frédéric Lavancier (1) , Paul Rochet (2)

1 Probabilités, statistique et calcul scientifique
2 LMJL - Laboratoire de Mathématiques Jean Leray

We propose a general method to combine several estimators of the same quantity in order to produce a better estimate. In the spirit of model and forecast averaging, the final estimator is computed as a weighted average of the initial ones, where the weights are constrained to sum to one. In this framework, the optimal weights, minimizing the quadratic loss, are entirely determined by the mean square error matrix of the vector of initial estimators. The averaging estimator is derived using an estimation of this matrix, which can be computed from the same dataset. We show that the solution satisfies a non-asymptotic oracle inequality and is asymptotically optimal, provided the mean square error matrix is suitably estimated. This method is illustrated on standard statistical problems: estimation of the position of a symmetric distribution, estimation in a parametric model, density estimation, where the averaging estimator outperforms the initial estimators in most cases.

Mots clés: Averaging; Aggregation; Oracle inequality; Parametric estimation; Weibull model; Kernel density estimation

Domaines: Méthodologie [stat.ME]

Frédéric Lavancier, Paul Rochet. A general procedure to combine estimators. 2014. ⟨hal-00936024v2⟩