Decomposition of the Rand index in order to assess both the stability and the number of clusters of a partition

Ghazi Bel Mufti (1) , Patrice Bertrand (2) , Lassad El Moubarki (3)

1 LARIME - Laboratoire de Recherche Interdisciplinaire sur les Mutations des Economies et des Entreprises [Tunis]
2 CEREMADE - CEntre de REcherches en MAthématiques de la DEcision
3 FSEG Sfax - Faculté des Sciences Economiques et de Gestion de Sfax

During the last decade, stability-based measures became popular in order to validate the results of partitioning methods. Among these measures, those are the simplest and most commonly used estimate the stability of a partition by performing a large number of comparisons between two partitions achieved on distinct perturbed sets. The Rand index provides a simple and easily interpretable way to achieve such comparisons. In this paper, we first propose an additive decomposition of a partitioning stability measure derived from the Rand index. Then, we derive a bi-criterion, which is based both on the isolation and on the cohesion of each individual cluster, in order to assess the optimal number of clusters of a partition. Based on simulated data sets, we compare our approach with the most successful methods for predicting the number of clusters.

Mots clés: Partition stability; cluster isolation; cluster cohesion; resampling; random noise

Domaines: Statistiques [math.ST]; Théorie [stat.TH]

Ghazi Bel Mufti, Patrice Bertrand, Lassad El Moubarki. Decomposition of the Rand index in order to assess both the stability and the number of clusters of a partition. 2012. ⟨hal-00707357⟩