Sparse Quantized Spectral Clustering

Zhenyu Liao (1) , Romain Couillet (2) , Michael W Mahoney (1)

1 UC Berkeley - University of California [Berkeley]
2 GIPSA-GAIA - GIPSA Pôle Géométrie, Apprentissage, Information et Algorithmes

Given a large data matrix, sparsifying, quantizing, and/or performing other entrywise nonlinear operations can have numerous benefits, ranging from speeding up iterative algorithms for core numerical linear algebra problems to providing nonlinear filters to design state-of-the-art neural network models. Here, we exploit tools from random matrix theory to make precise statements about how the eigenspectrum of a matrix changes under such nonlinear transformations. In particular, we show that very little change occurs in the informative eigenstructure, even under drastic sparsification/quantization, and consequently that very little downstream performance loss occurs when working with very aggressively sparsified or quantized spectral clustering problems. We illustrate how these results depend on the nonlinearity, we characterize a phase transition beyond which spectral clustering becomes possible, and we show when such nonlinear transformations can introduce spurious non-informative eigenvectors.

Domaines: Statistiques [math.ST]; Apprentissage [cs.LG]; Traitement du signal et de l'image [eess.SP]

Zhenyu Liao, Romain Couillet, Michael W Mahoney. Sparse Quantized Spectral Clustering. ICLR 2021 - 9th International Conference on Learning Representations, May 2021, Virtual Only, France. ⟨hal-03114509⟩