Construction of an off-centered entropy for supervised learning

Stéphane Lallich (1) , Philippe Lenca (2, 3, 4) , Benoît Vaillant (5)

1 ERIC - Equipe de Recherche en Ingénierie des Connaissances
2 Lab-STICC_IMTA_CID_DECIDE
3 LUSSI - Département Logique des Usages, Sciences sociales et Sciences de l'Information
4 TAMCIC - Traitement Algorithmique et Matériel de la Communication, de l'Information et de la Connaissance
5 VALORIA - Laboratoire de recherche en informatique et ses applications [Vannes]

In supervised learning, many measures are based on the concept of entropy. A major characteristic of the entropies is that they take their maximal value when the distribution of the modalities of the class variable is uniform. To deal with the case where the a priori frequencies of the class variable modalities are very imbalanced, we propose an off-centered entropy which takes its maximum value for a distribution fixed by the user. This distribution can be the a priori distribution of the class variable modalities or a distribution taking into account the costs of misclassification.

Domaines: Ordinateur et société [cs.CY]

Stéphane Lallich, Philippe Lenca, Benoît Vaillant. Construction of an off-centered entropy for supervised learning. ASMDA 2007 : XIIth International Symposium on Applied Stochastic Models and Data Analysis, May 29 - June 1, Chania, Crete, Greece, May 2007, Crete, Greece. ⟨hal-02121319⟩