Impact of subsampling and tree depth on random forests

Abstract : Random forests are ensemble learning methods introduced by Breiman (2001) that operate by averaging several decision trees built on a randomly selected subspace of the data set. Despite their widespread use in practice, the respective roles of the different mechanisms at work in Breiman's forests are not yet fully understood, neither is the tuning of the corresponding parameters. In this paper, we study the influence of two parameters, namely the subsampling rate and the tree depth, on Breiman's forests performance. More precisely, we prove that quantile forests (a specific type of random forests) based on subsampling and quantile forests whose tree construction is terminated early have similar performances, as long as their respective parameters (subsampling rate and tree depth) are well chosen. Moreover, experiments show that a proper tuning of these parameters leads in most cases to an improvement of Breiman's original forests in terms of mean squared error.
Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-01287521
Contributor : Erwan Scornet <>
Submitted on : Tuesday, May 7, 2019 - 10:29:20 AM
Last modification on : Sunday, May 26, 2019 - 1:16:11 AM

Files


Identifiers

  • HAL Id : hal-01287521, version 3
  • ARXIV : 1603.04261

Citation

Roxane Duroux, Erwan Scornet. Impact of subsampling and tree depth on random forests. 2016. ⟨hal-01287521v3⟩

Share

Metrics

Record views

26

Files downloads

77