Local Q-Linear Convergence and Finite-time Active Set Identification of ADMM on a Class of Penalized Regression Problems

Abstract : We study the convergence of the ADMM (Alternating Direction Method of Multipliers) algorithm on a broad range of penalized regression problems including the Lasso, Group-Lasso and Graph-Lasso,(isotropic) TV-L1, Sparse Variation, and others. First, we establish a fixed-point iteration –via a nonlinear operator– which is equivalent to the ADMM iterates. We then show that this nonlinear operator is Fréchet-differentiable almost everywhere and that around each fixed point, Q-linear convergence is guaranteed, provided the spectral radius of the Jacobian of the operator at the fixed point is less than 1 (a classical result on stability). Moreover, this spectral radius is then a rate of convergence for the ADMM algorithm. Also, we show that the support of the split variable can be identified after finitely many iterations. In the anisotropic cases, we show that for sufficiently large values of the tuning parameter, we recover the optimal rates in terms of Friedrichs angles, that have appeared recently in the literature. Empirical results on various problems are also presented and discussed.
Liste complète des métadonnées

Cited literature [27 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01265372
Contributor : Elvis Dohmatob <>
Submitted on : Monday, February 1, 2016 - 9:12:40 AM
Last modification on : Friday, March 8, 2019 - 1:20:22 AM
Document(s) archivé(s) le : Friday, November 11, 2016 - 10:22:15 PM

File

paper.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01265372, version 1

Citation

Elvis Dohmatob, Michael Eickenberg, Bertrand Thirion, Gaël Varoquaux. Local Q-Linear Convergence and Finite-time Active Set Identification of ADMM on a Class of Penalized Regression Problems. ICASSP, International Conference on Acoustics, Speach, and Signal Processing, Mar 2016, http://icassp2016.org/default.asp, China. ⟨hal-01265372⟩

Share

Metrics

Record views

433

Files downloads

408