A constrained singular value decomposition method that integrates sparsity and orthogonality

Vincent Guillemot (1) , Derek Beaton , Arnaud Gloaguen (2) , Tommy Lofstedt (3) , Brian Levine , Nicolas Raymond (4) , Arthur Tenenhaus (2) , Hervé Abdi (5)

1 Hub Bioinformatique et Biostatistique - Bioinformatics and Biostatistics HUB
2 L2S - Laboratoire des signaux et systèmes
3 Umeå University
4 IRMAR - Institut de Recherche Mathématique de Rennes
5 University of Texas at Austin [Austin]

We propose a new sparsification method for the singular value decomposition—called the constrained singular value decomposition (CSVD)—that can incorporate multiple constraints such as sparsification and orthogonality for the left and right singular vectors. The CSVD can combine different constraints because it implements each constraint as a projection onto a convex set, and because it integrates these constraints as projections onto the intersection of multiple convex sets. We show that, with appropriate sparsification constants, the algorithm is guaranteed to converge to a stable point. We also propose and analyze the convergence of an efficient algorithm for the specific case of the projection onto the balls defined by the norms L1 and L2. We illustrate the CSVD and compare it to the standard singular value decomposition and to a non-orthogonal related sparsification method with: 1) a simulated example, 2) a small set of face images (corresponding to a configuration with a number of variables much larger than the number of observations), and 3) a psychometric application with a large number of observations and a small number of variables. The companion R-package, csvd, that implements the algorithms described in this paper, along with reproducible examples, are available for download from https://github.com/vguillemot/csvd.

Mots clés: Psychometrics; Optimization; Vector spaces; Algorithms; Singular value decomposition; Eigenvalues; Principal component analysis; Simulation and modeling

Domaines: Optimisation et contrôle [math.OC]

Vincent Guillemot, Derek Beaton, Arnaud Gloaguen, Tommy Lofstedt, Brian Levine, et al.. A constrained singular value decomposition method that integrates sparsity and orthogonality. PLoS ONE, 2019, 14 (3), pp.e0211463. ⟨10.1371/journal.pone.0211463⟩. ⟨hal-02078111⟩