Gradient boosting for kernelized output spaces

Pierre Geurts (1) , Louis Wehenkel (2) , Florence d'Alché-Buc (1)

1 IBISC - Informatique, Biologie Intégrative et Systèmes Complexes
2 Institut Montefiore - Department of Electrical Engineering and Computer Science

A general framework is proposed for gradient boosting in supervised learning problems where the loss function is defined using a kernel over the output space. It extends boosting in a principled way to complex output spaces (images, text, graphs etc.) and can be applied to a general class of base learners working in kernelized output spaces. Empirical results are provided on three problems: a regression problem, an image completion task and a graph prediction problem. In these experiments, the framework is combined with tree-based base learners, which have interesting algorithmic properties. The results show that gradient boosting significantly improves these base learners and provides competitive results with other tree-based ensemble methods based on randomization.

Domaines: Informatique; Bio-informatique [q-bio.QM]; Bio-Informatique, Biologie Systémique [q-bio.QM]; Modélisation et simulation; Apprentissage [cs.LG]; Base de données [cs.DB]

Pierre Geurts, Louis Wehenkel, Florence d'Alché-Buc. Gradient boosting for kernelized output spaces. Proceedings of the 24th Annual International Conference on Machine Learning (ICML 2007), 2007, Corvallis, OR, United States. pp.289--296, ⟨10.1145/1273496.1273533⟩. ⟨hal-00341945⟩