Boosting and Instability for Regression Trees

Servane Gey (1) , Jean-Michel Poggi (2)

1 MAP5 - UMR 8145 - Mathématiques Appliquées Paris 5
2 LM-Orsay - Laboratoire de Mathématiques d'Orsay

The AdaBoost like algorithm for boosting CART regression trees is considered. The boosting predictors sequence is analyzed on various data sets and the behaviour of the algorithm is investigated. An instability index of a given estimation method with respect to some training sample is defined. Based on the bagging algorithm, this instability index is then extended to quantify the additional instability provided by the boosting process with respect to the bagging one. Finally, the ability of boosting to track outliers and to concentrate on hard observations is used to explore a non-standard regression context.

Mots clés: Bagging; Boosting; CART; Instability; Prediction; Regression

Domaines: Statistiques [math.ST]

Servane Gey, Jean-Michel Poggi. Boosting and Instability for Regression Trees. Computational Statistics and Data Analysis, 2006, 50 (2), pp.533 - 550. ⟨hal-00326558⟩