Local Q-Linear Convergence and Finite-time Active Set Identification of ADMM on a Class of Penalized Regression Problems

Abstract : We study the convergence of the ADMM (Alternating Direction Method of Multipliers) algorithm on a broad range of penalized regression problems including the Lasso, Group-Lasso and Graph-Lasso,(isotropic) TV-L1, Sparse Variation, and others. First, we establish a fixed-point iteration –via a nonlinear operator– which is equivalent to the ADMM iterates. We then show that this nonlinear operator is Fréchet-differentiable almost everywhere and that around each fixed point, Q-linear convergence is guaranteed, provided the spectral radius of the Jacobian of the operator at the fixed point is less than 1 (a classical result on stability). Moreover, this spectral radius is then a rate of convergence for the ADMM algorithm. Also, we show that the support of the split variable can be identified after finitely many iterations. In the anisotropic cases, we show that for sufficiently large values of the tuning parameter, we recover the optimal rates in terms of Friedrichs angles, that have appeared recently in the literature. Empirical results on various problems are also presented and discussed.
Type de document :
Communication dans un congrès
ICASSP, International Conference on Acoustics, Speach, and Signal Processing, Mar 2016, http://icassp2016.org/default.asp, China. ICASSP, International Conference on Acoustics, Speach, and Signal Processing, ICASSP, International Conference on Acoustics, Speach, and Signal Processing
Liste complète des métadonnées

Littérature citée [27 références]  Voir  Masquer  Télécharger

https://hal.archives-ouvertes.fr/hal-01265372
Contributeur : Elvis Dohmatob <>
Soumis le : lundi 1 février 2016 - 09:12:40
Dernière modification le : jeudi 7 février 2019 - 15:48:28
Document(s) archivé(s) le : vendredi 11 novembre 2016 - 22:22:15

Fichier

paper.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01265372, version 1

Citation

Elvis Dohmatob, Michael Eickenberg, Bertrand Thirion, Gaël Varoquaux. Local Q-Linear Convergence and Finite-time Active Set Identification of ADMM on a Class of Penalized Regression Problems. ICASSP, International Conference on Acoustics, Speach, and Signal Processing, Mar 2016, http://icassp2016.org/default.asp, China. ICASSP, International Conference on Acoustics, Speach, and Signal Processing, ICASSP, International Conference on Acoustics, Speach, and Signal Processing. 〈hal-01265372〉

Partager

Métriques

Consultations de la notice

425

Téléchargements de fichiers

395