Pruning Random Forest with Orthogonal Matching Trees

Luc Giffon (1) , Charly Lamothe (1) , Léo Bouscarrat (1) , Paolo Milanesi (2, 1) , Farah Cherfaoui (1, 2) , Sokol Koço (1)

1 QARMA - éQuipe d'AppRentissage de MArseille
2 I2M - Institut de Mathématiques de Marseille

In this paper we propose a new method to reduce the size of Breiman's Random Forests. Given a Random Forest and a target size, our algorithm builds a linear combination of trees which minimizes the training error. Selected trees, as well as weights of the linear combination are obtained by mean of the Orthogonal Matching Pursuit algorithm. We test our method on many public benchmark datasets both on regression and binary classification and we compare it to other pruning techniques. Experiments show that our technique performs significantly better or equally good on many datasets 1. We also discuss the benefit and shortcoming of learning weights for the pruned forest which lead us to propose to use a non-negative constraint on the OMP weights for better empirical results.

Mots clés: random forest; pruning; OMP

Domaines: Apprentissage [cs.LG]

Luc Giffon, Charly Lamothe, Léo Bouscarrat, Paolo Milanesi, Farah Cherfaoui, et al.. Pruning Random Forest with Orthogonal Matching Trees. https://cap-rfiap2020.sciencesconf.org/, Jun 2020, Vannes, France. ⟨hal-02534421⟩