Structure-Adaptive Accelerated Coordinate Descent

Junqi Tang 1 Mohammad Golbabaee 2 Francis Bach 3, 4 Mike Davies 1
4 SIERRA - Statistical Machine Learning and Parsimony
DI-ENS - Département d'informatique de l'École normale supérieure, CNRS - Centre National de la Recherche Scientifique, Inria de Paris
Abstract : In this work we explore the fundamental structure-adaptiveness of accelerated random-ized coordinate descent algorithms on regular-ized empirical risk minimization tasks, where the solution has intrinsic low-dimensional structure such as sparsity and low-rank, enforced by non-smooth regularization. We propose and analyze a two-stage accelerated coordinate descent algorithm ("two-stage APCG") utilizing the restricted strong-convexity framework. We provide the convergence analysis showing that the proposed method have a local accelerated linear convergence rate with respect to the low-dimensional structure of the solution. We also propose an adaptive variant of the two-stage APCG which does not need to foreknow the restricted strong con-vexity parameter beforehand, but estimates it on the fly. In our numerical experiments we test the proposed method on a number of machine learning datasets and demonstrate the effectiveness of our approach.
Document type :
Preprints, Working Papers, ...
Complete list of metadatas

Cited literature [5 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01889990
Contributor : Junqi Tang <>
Submitted on : Monday, October 8, 2018 - 11:38:00 AM
Last modification on : Thursday, December 13, 2018 - 8:02:02 PM
Long-term archiving on: Wednesday, January 9, 2019 - 2:06:55 PM

File

TSAPCG_1.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01889990, version 1

Citation

Junqi Tang, Mohammad Golbabaee, Francis Bach, Mike Davies. Structure-Adaptive Accelerated Coordinate Descent. 2018. ⟨hal-01889990v1⟩

Share

Metrics

Record views

61

Files downloads

32