Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

VARCLUST: clustering variables using dimensionality reduction

Abstract : VARCLUST algorithm is proposed for clustering variables under the assumption that variables in a given cluster are linear combinations of a small number of hidden latent variables, corrupted by the random noise. The entire clustering task is viewed as the problem of selection of the statistical model, which is defined by the number of clusters, the partition of variables into these clusters and the 'cluster dimensions', i.e. the vector of dimensions of linear subspaces spanning each of the clusters. The "optimal" model is selected using the approximate Bayesian criterion based on the Laplace approximations and using a non-informative uniform prior on the number of clusters. To solve the problem of the search over a huge space of possible models we propose an extension of the ClustOfVar algorithm of [29, 7] which was dedicated to subspaces of dimension only 1, and which is similar in structure to the K-centroid algorithm. We provide a complete methodology with theoretical guarantees, extensive numerical experi-mentations, complete data analyses and implementation. Our algorithm assigns variables to appropriate clusterse based on the consistent Bayesian Information Criterion (BIC), and estimates the dimensionality of each cluster by the PEnalized SEmi-integrated Likelihood Criterion (PESEL) of [24], whose consistency we prove. Additionally, we prove that each iteration of our algorithm leads to an increase of the Laplace approximation to the model posterior probability and provide the criterion for the estimation of the number of clusters. Numerical comparisons with other algorithms show that VARCLUST may outperform some popular machine learning tools for sparse subspace clustering. We also report the results of real data analysis including TCGA breast cancer data and meteorological data, which show that the algorithm can lead to meaningful clustering. The proposed method is implemented in the publicly available R package varclust. Keywords variable clustering · Bayesian approach · k-means · dimensionality reduction · subspace clustering 2 P. Sobczyk, S. Wilczyński, M. Bogdan et al.
Document type :
Preprints, Working Papers, ...
Complete list of metadata
Contributor : Fabien Panloup Connect in order to contact the contributor
Submitted on : Friday, December 18, 2020 - 3:33:45 PM
Last modification on : Wednesday, November 3, 2021 - 6:05:01 AM


Files produced by the author(s)


  • HAL Id : hal-03002017, version 2



Piotr Sobczyk, Stanislaw Wilczynski, Malgorzata Bogdan, Piotr Graczyk, Julie Josse, et al.. VARCLUST: clustering variables using dimensionality reduction. 2020. ⟨hal-03002017v2⟩



Record views


Files downloads