The Geometry of Uniqueness, Sparsity and Clustering in Penalized Estimation

Ulrike Schneider (1) , Patrick A Tardivel (2, 3)

1 TU Wien - Vienna University of Technology = Technische Universität Wien
2 UWr - University of Wrocław [Poland]
3 UB - Université de Bourgogne

We provide a necessary and sufficient condition for the uniqueness of penalized least-squares estimators whose penalty term is given by a norm with a polytope unit ball, covering a wide range of methods including SLOPE and LASSO, as well as the related method of basis pursuit. We consider a strong type of uniqueness that is relevant for statistical problems. The uniqueness condition is geometric and involves how the row span of the design matrix intersects the faces of the dual norm unit ball, which for SLOPE is given by the sign permutahedron. Further considerations based this condition also allow to derive results on sparsity and clustering features. In particular, we define the notion of a SLOPE model to describe both sparsity and clustering properties of this method and also provide a geometric characterization of accessible SLOPE models.

Domaines: Mathématiques [math]; Statistiques [math.ST]

Ulrike Schneider, Patrick A Tardivel. The Geometry of Uniqueness, Sparsity and Clustering in Penalized Estimation. Journal of Machine Learning Research, In press. ⟨hal-02548350v3⟩