Asaga: Asynchronous Parallel Saga

Rémi Leblond 1, 2 Fabian Pedregosa 1, 2 Simon Lacoste-Julien 1, 2, 3
1 SIERRA - Statistical Machine Learning and Parsimony
DI-ENS - Département d'informatique de l'École normale supérieure, CNRS - Centre National de la Recherche Scientifique, Inria de Paris
Abstract : We describe Asaga, an asynchronous parallel version of the incremental gradient algorithm Saga that enjoys fast linear convergence rates. We highlight a subtle but important technical issue present in a large fraction of the recent convergence rate proofs for asynchronous parallel optimization algorithms, and propose a simplification of the recently proposed " perturbed iterate " framework that resolves it. We thereby prove that Asaga can obtain a theoretical linear speedup on multi-core systems even without sparsity assumptions. We present results of an implementation on a 40-core architecture illustrating the practical speedup as well as the hardware overhead.
Document type :
Preprints, Working Papers, ...
Complete list of metadatas

Cited literature [23 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01407833
Contributor : Rémi Leblond <>
Submitted on : Friday, December 2, 2016 - 3:39:42 PM
Last modification on : Tuesday, January 29, 2019 - 3:10:33 PM
Long-term archiving on: Thursday, March 23, 2017 - 12:10:35 AM

File

1606.04809v1.pdf
Files produced by the author(s)

Licence


Copyright

Identifiers

Collections

Citation

Rémi Leblond, Fabian Pedregosa, Simon Lacoste-Julien. Asaga: Asynchronous Parallel Saga. 2016. ⟨hal-01407833⟩

Share

Metrics

Record views

511

Files downloads

114