Online Non-Linear Gradient Boosting in Multi-Latent Spaces

Jordan Frery (1) , Amaury Habrard (1) , Marc Sebban (1) , Olivier Caelen (2) , Liyun He-Guelton (3)

1 LHC - Laboratoire Hubert Curien
2 Atos Worldline
3 IFREMER - Institut Français de Recherche pour l'Exploitation de la Mer

Gradient Boosting is a popular ensemble method that combines linearly diverse and weak hypotheses to build a strong classifier. In this work, we propose a new Online Non-Linear gradient Boosting (ONLB) algorithm where we suggest to jointly learn different combinations of the same set of weak classifiers in order to learn the idiosyncrasies of the target concept. To expand the expressiveness of the final model, our method leverages the non linear complementarity of these combinations. We perform an experimental study showing that ONLB (i) outperforms most recent online boosting methods in both terms of convergence rate and accuracy and (ii) learns diverse and useful new latent spaces.

Domaines: Apprentissage [cs.LG]

Jordan Frery, Amaury Habrard, Marc Sebban, Olivier Caelen, Liyun He-Guelton. Online Non-Linear Gradient Boosting in Multi-Latent Spaces. 17th International Symposium on Intelligent Data Analysis (IDA'18), Oct 2018, s-Hertogenbosch, Netherlands. ⟨hal-01889994⟩