Ensemble Feature Ranking

Kees Jong , Jérémie Mary (1) , Antoine Cornuéjols (2) , Elena Marchiori (3) , Michèle Sebag (4)

1 CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189
2 MIA-Paris - Mathématiques et Informatique Appliquées
3 IBIVU, - IBIVU, Department of Computer Science, Faculty of Sciences
4 LRI - Laboratoire de Recherche en Informatique

A crucial issue for Machine Learning and Data Mining is Feature Selection, selecting the relevant features in order to focus the learning search. A relaxed setting for Feature Selection is known as Feature Ranking, ranking the features with respect to their relevance. This paper proposes an ensemble approach for Feature Ranking, aggre-gating feature rankings extracted along independent runs of an evolutionary learning algorithm named ROGER. The convergence of ensemble feature ranking is studied in a theoretical perspective, and a statistical model is devised for the empirical validation, inspired from the complexity framework proposed in the Constraint Satisfaction domain. Comparative experiments demonstrate the robustness of the approach for learning (a limited kind of) non-linear concepts, specifically when the features significantly outnumber the examples.

Domaines: Apprentissage [cs.LG]; Intelligence artificielle [cs.AI]

Kees Jong, Jérémie Mary, Antoine Cornuéjols, Elena Marchiori, Michèle Sebag. Ensemble Feature Ranking. ECML-PKDD, Sep 2004, Pisa, Italy. ⟨hal-02482166⟩