Relationship between the optimal solutions of least squares regularized with L0-norm and constrained by k-sparsity

Mila Nikolova (1)

1 CMLA - Centre de Mathématiques et de Leurs Applications

The recovery of a sparse vector from a noisy underdetermined linear system arises in various fields. Two desirable models to find a sparse solution are the constrained problem where the quadratic error is minimized subject to a given level of sparsity and the regularized problem where both the quadratic error and the sparsity are minimized using a regularization parameter. The existence of connections between these two problems is intuitive. However, the mechanism of this relation has remained unclear so far. This work provides an exhaustive description of the relationship between the globally optimal solutions of these two problems. A partial equivalence between them always exists. We exhibit formulae for a sequence of critical parameters that can enable equality between the global minimizers of these problems. This sequence always admits a strictly decreasing subsequence that partitions the positive axis into a certain number of intervals. For every value of the regularization parameter inside an interval, there is a sparsity level k such that the global minimizers of the regularized problem and those of the kconstrained problem coincide. At the values of the subsequence of critical parameters, the optimal set of the regularized problem contains two optimal sets of the constrained problem. In some cases the whole sequence of critical parameters is strictly decreasing. Then the number of the intervals on the positive axis equals the number of all sparsity levels and both problems are quasi-completely equivalent. The critical parameters are obtained from the optimal values of the constrained problem. Examples and small-size exact numerical tests are given to illustrate our theoretical results. Our contributions yield various open questions and can help the design of innovative numerical schemes.

Mots clés: L0-regularization; k-sparsity constraint; quasi-equivalence between nonconvex problems; optimal solution analysis; parameter selection; sparse signal recovery; under-determined linear systems; globally optimal solutions

Domaines: Analyse numérique [math.NA]; Traitement du signal et de l'image [eess.SP]; Traitement du signal et de l'image [eess.SP]

Mila Nikolova. Relationship between the optimal solutions of least squares regularized with L0-norm and constrained by k-sparsity. Applied and Computational Harmonic Analysis, 2015, ⟨10.1016/j.acha.2015.10.010⟩. ⟨hal-00944006v3⟩