A multi-model selection framework for unknown and/or evolutive misclassification cost problems

Clement Chatelain (1) , Sébastien Adam (1) , Yves Lecourtier (1) , Laurent Heutte (1) , Thierry Paquet (1)

1 LITIS - Laboratoire d'Informatique, de Traitement de l'Information et des Systèmes

In this paper, we tackle the problem of model selection when misclassification costs are unknown and/or may evolve. Unlike traditional approaches based on a scalar optimization, we propose a generic multi-model selection framework based on a multi-objective approach. The idea is to automatically train a pool of classifiers instead of one single classifier, each classifier in the pool optimizing a particular trade-off between the objectives. Within the context of two-class classification problems, we introduce the "ROC front concept" as an alternative to the ROC curve representation. This strategy is applied to the multi-model selection of SVM classifiers using an evolutionary multi-objective optimization algorithm. The comparison with a traditional scalar optimization technique based on an AUC criterion shows promising results on UCI datasets as well as on a real-world classification problem.

Domaines: Traitement du texte et du document

Clement Chatelain, Sébastien Adam, Yves Lecourtier, Laurent Heutte, Thierry Paquet. A multi-model selection framework for unknown and/or evolutive misclassification cost problems. Pattern Recognition, 2010, 43 (3), pp.815-823. ⟨hal-00435954⟩