A thorough experimental study of datasets for frequent itemsets

Frédéric Flouvat (1) , Fabien de Marchi (2) , Jean-Marc Petit (2)

1 LIMOS - Laboratoire d'Informatique, de Modélisation et d'Optimisation des Systèmes
2 BD - Base de Données

The discovery of frequent patterns is a famous problem in data mining. While plenty of algorithms have been proposed during the last decade, only a few contributions have tried to understand the influence of datasets on the algorithms behavior. Being able to explain why certain algorithms are likely to perform very well or very poorly on some datasets is still an open question. In this setting, we describe a thorough experimental study of datasets with respect to frequent itemsets. We study the distribution of frequent itemsets with respect to itemsets size together with the distribution of three concise representations: frequent closed, frequent free and frequent essential itemsets. For each of them, we also study the distribution of their positive and negative borders whenever possible. From this analysis, we exhibit a new characterization of datasets and some invariants allowing to better predict the behavior of well known algorithms. The main perspective of this work is to devise adaptive algorithms with respect to dataset characteristics.

Domaines: Informatique [cs]

Frédéric Flouvat, Fabien de Marchi, Jean-Marc Petit. A thorough experimental study of datasets for frequent itemsets. International Conference of Data Mining (ICDM'05), Nov 2005, Houston, Texas, United States. pp.162-169. ⟨hal-01590948⟩