Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Structure-Adaptive Accelerated Coordinate Descent

Junqi Tang 1 Mohammad Golbabaee 2 Francis Bach 3, 4 Mike Davies 1
4 SIERRA - Statistical Machine Learning and Parsimony
DI-ENS - Département d'informatique de l'École normale supérieure, CNRS - Centre National de la Recherche Scientifique, Inria de Paris
Abstract : In this work we explore the fundamental structure-adaptiveness of accelerated randomized coordinate descent algorithms on regularized empirical risk minimization tasks, where the solution has intrinsic low-dimensional structure such as sparsity and low-rank, enforced by non-smooth regularization. We propose and analyze a two-stage accelerated coordinate descent algorithm ("two-stage APCG") utilizing the restricted strong-convexity framework. We provide the convergence analysis showing that the proposed method have a local accelerated linear convergence rate with respect to the low-dimensional structure of the solution. We also propose an adaptive variant of the two-stage APCG which does not need to foreknow the restricted strong convexity parameter beforehand, but estimates it on the fly. In our numerical experiments we test the proposed method on a number of machine learning datasets and demonstrate the effectiveness of our approach.
Document type :
Preprints, Working Papers, ...
Complete list of metadata
Contributor : Junqi Tang <>
Submitted on : Wednesday, October 17, 2018 - 3:23:22 PM
Last modification on : Tuesday, May 4, 2021 - 2:06:02 PM
Long-term archiving on: : Friday, January 18, 2019 - 2:26:19 PM


Files produced by the author(s)


  • HAL Id : hal-01889990, version 2



Junqi Tang, Mohammad Golbabaee, Francis Bach, Mike Davies. Structure-Adaptive Accelerated Coordinate Descent. 2018. ⟨hal-01889990v2⟩



Record views


Files downloads