Using big steps in coordinate descent primal-dual algorithms

Pascal Bianchi (1, 2) , Olivier Fercoq (1, 2)

1 LTCI - Laboratoire Traitement et Communication de l'Information
2 S2A - Signal, Statistique et Apprentissage

The Vu-Condat algorithm is a standard method for finding a saddle point of a Lagrangian involving a differentiable function. Recent works have tried to adapt the idea of random coordinate descent to this algorithm, with the aim to efficiently solve some regularized or distributed optimization problems. A drawback of these approaches is that the admissible step sizes can be small, leading to slow convergence. In this paper, we introduce a coordinate descent primal-dual algorithm which is provably convergent for a wider range of step size values than previous methods. In particular, the condition on the step-sizes depends on the coordinate-wise Lipschitz constant of the differentiable function's gradient. We discuss the application of our method to distributed optimization and large scale support vector machine problems.

Domaines: Optimisation et contrôle [math.OC]

Pascal Bianchi, Olivier Fercoq. Using big steps in coordinate descent primal-dual algorithms. IEEE 55th Conference on Decision and Control (CDC), Dec 2016, Las Vegas, NV, United States. pp.1895-1899, ⟨10.1109/CDC.2016.7798541⟩. ⟨hal-01497087⟩