COBRA: A combined regression strategy

Gérard Biau (1) , Aurélie Fischer (2) , Benjamin Guedj (3) , James D. Malley (4)

1 LSTA - Laboratoire de Statistique Théorique et Appliquée
2 LPMA - Laboratoire de Probabilités et Modèles Aléatoires
3 MODAL - MOdel for Data Analysis and Learning
4 NIH - National Institutes of Health [Bethesda, MD, USA]

A new method for combining several initial estimators of the regression function is introduced. Instead of building a linear or convex optimized combination over a collection of basic estimators r1,…,rMr1,…,rM, we use them as a collective indicator of the proximity between the training data and a test observation. This local distance approach is model-free and very fast. More specifically, the resulting nonparametric/nonlinear combined estimator is shown to perform asymptotically at least as well in the L2L2 sense as the best combination of the basic estimators in the collective. A companion R package called COBRA (standing for COmBined Regression Alternative) is presented (downloadable on http://cran.r-project.org/web/packages/COBRA/index.html). Substantial numerical evidence is provided on both synthetic and real data sets to assess the excellent performance and velocity of our method in a large variety of prediction problems.

Domaines: Probabilités [math.PR]

Gérard Biau, Aurélie Fischer, Benjamin Guedj, James D. Malley. COBRA: A combined regression strategy. Journal of Multivariate Analysis, 2016, 146, pp.18-28. ⟨10.1016/j.jmva.2015.04.007⟩. ⟨hal-01361789v1⟩