Tree-based ranking methods

Stéphan Clémençon (1) , Nicolas Vayatis (2)

1 LTCI - Laboratoire Traitement et Communication de l'Information
2 CMLA - Centre de Mathématiques et de Leurs Applications

Recursive partitioning methods are among the most popular techniques in machine learning. The paper investigates how these methods can be adapted to the bipartite ranking problem. In ranking, the pursued goal is global: based on past data, define an order on the whole input space, so that positive instances take up the top ranks with maximum probability. The most natural way to order all instances consists of projecting the input data onto the real line through a real-valued scoring function and use the natural order on R. The accuracy of the ordering induced by a candidate s is classically measured in terms of the ROC curve or the area under the ROC curve (AUC). Here we discuss the design of tree-structured scoring functions obtained by recursively maximizing the AUC criterion. A novel tree-based algorithm, called TreeRank, specifically designed for learning to rank/order instances is proposed. Consistency results and generalization bounds of functional nature are established for this ranking method, when considering either the L1 or supremum norm. Inspired from recent developments in the field of binary classification, we also describe committee-based learning procedures using TreeRank as a "base ranker", in order to overcome obvious drawbacks of such a top-down partitioning technique. Preliminary simulation results are also displayed.

Mots clés: ranking; decision trees; recursive partitioning; consistency; auc

Domaines: Statistiques [math.ST]; Théorie [stat.TH]

Stéphan Clémençon, Nicolas Vayatis. Tree-based ranking methods. 2008. ⟨hal-00268068v4⟩