HAL : hal-00413154, version 1

arXiv : 0909.0635

DOI : 10.1007/978-3-642-01805-3_4

Fiche détaillée

Récupérer au format BibTeX - EndNote - TEI - RefWorks -

Similarity-Based Clustering, Villmann, Th.; Biehl, M.; Hammer, B.; Verleysen, M. (Ed.) (2009) 52-69

Advances in Feature Selection with Mutual Information

Michel Verleysen 1, Fabrice Rossi (  Auteur correspondant ) 2, Damien François 3

(05/2009)

The selection of features that are relevant for a prediction or classification problem is an important problem in many domains involving high-dimensional data. Selecting features helps fighting the curse of dimensionality, improving the performances of prediction or classification methods, and interpreting the application. In a nonlinear context, the mutual information is widely used as relevance criterion for features and sets of features. Nevertheless, it suffers from at least three major limitations: mutual information estimators depend on smoothing parameters, there is no theoretically justified stopping criterion in the feature selection greedy procedure, and the estimation itself suffers from the curse of dimensionality. This chapter shows how to deal with these problems. The two first ones are addressed by using resampling techniques that provide a statistical basis to select the estimator parameters and to stop the search procedure. The third one is addressed by modifying the mutual information criterion into a measure of how features are complementary (and not only informative) for the problem at hand.

1 :	Dispositifs Intégrés et Circuits Electroniques Machine Learning Group (DICE - MLG)
	Université Catholique de Louvain
2 :	Laboratoire traitement et communication de l'information (LTCI)
	CNRS : UMR5141 – Institut Télécom – Télécom ParisTech
3 :	Centre for Systems Engineering and Applied Mechanics (CSAM)
	Université Catholique de Louvain

Domaine

:

Informatique/Apprentissage Informatique/Théorie de l'information et codage Mathématiques/Théorie de l'information et codage

Mots Clés : Feature Selection – Mutual Information – Resampling – Forward selection

hal-00413154, version 1

http://hal.archives-ouvertes.fr/hal-00413154

oai:hal.archives-ouvertes.fr:hal-00413154

Contributeur : Fabrice Rossi <>

Soumis le : Jeudi 3 Septembre 2009, 12:37:03

Dernière modification le : Jeudi 3 Septembre 2009, 14:05:14