An Efficient Algorithm for Computing Entropic Measures of Feature Subsets

Frédéric Pennerath (1, 2)

1 CentraleSupélec
2 ORPAILLEUR - Knowledge representation, reasonning

Entropic measures such as conditional entropy or mutual information have been used numerous times in pattern mining, for instance to characterize valuable itemsets or approximate functional dependencies. Strangely enough the fundamental problem of designing efficient algorithms to compute entropy of subsets of features (or mutual information of feature subsets relatively to some target feature) has received little attention compared to the analog problem of computing frequency of itemsets. The present article proposes to fill this gap: it introduces a fast and scalable method that computes entropy and mutual information for a large number of feature subsets by adopting the divide and conquer strategy used by FP-growth-one of the most efficient frequent itemset mining algorithm. In order to illustrate its practical interest, the algorithm is then used to solve the recently introduced problem of mining reliable approximate functional dependencies. It finally provides empirical evidences that in the context of non-redundant pattern extraction, the proposed method outperforms existing algorithms for both speed and scalability.

Mots clés: Pattern mining; Pattern redundancy; Entropic measures; Algorithm efficiency; Approximate Functional Dependency

Domaines: Apprentissage [cs.LG]; Machine Learning [stat.ML]; Théorie de l'information [cs.IT]; Algorithme et structure de données [cs.DS]

Frédéric Pennerath. An Efficient Algorithm for Computing Entropic Measures of Feature Subsets. ECML-PKDD 2018 - European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, Sep 2018, Dublin, Ireland. ⟨hal-01897734⟩