Kernelizing the output of tree-based methods

Pierre Geurts (1) , Louis Wehenkel (2) , Florence d'Alché-Buc (1)

1 IBISC - Informatique, Biologie Intégrative et Systèmes Complexes
2 Institut Montefiore - Department of Electrical Engineering and Computer Science

We extend tree-based methods to the prediction of structured outputs using a kernelization of the algorithm that allows one to grow trees as soon as a kernel can be defined on the output space. The resulting algorithm, called output kernel trees (OK3), generalizes classification and regression trees as well as tree-based ensemble methods in a principled way. It inherits several features of these methods such as interpretability, robustness to irrelevant variables, and input scalability. When only the Gram matrix over the outputs of the learning sample is given, it learns the output kernel as a function of inputs. We show that the proposed algorithm works well on an image reconstruction task and on a biological network inference problem.

Domaines: Informatique; Bio-informatique [q-bio.QM]; Bio-Informatique, Biologie Systémique [q-bio.QM]; Modélisation et simulation; Apprentissage [cs.LG]; Base de données [cs.DB]

Pierre Geurts, Louis Wehenkel, Florence d'Alché-Buc. Kernelizing the output of tree-based methods. Proc. of the 23rd International Conference on Machine Learning, 2006, United States. pp.345--352, ⟨10.1145/1143844.1143888⟩. ⟨hal-00341946⟩