Asynchronous decentralized convex optimization through short-term gradient averaging

Jerome Fellus (1) , David Picard (2) , Philippe-Henri Gosselin (2)

1 MIDI - Multimedia Indexation and Data Integration
2 ETIS - UMR 8051 - Equipes Traitement de l'Information et Systèmes

This paper considers decentralized convex optimization over a network in large scale contexts, where large simultaneously applies to number of training examples, dimensionality and number of networking nodes. We first propose a centralized optimization scheme that generalizes successful existing methods based on gradient averaging, improving their flexibility by making the number of averaged gradients an explicit parameter of the method. We then propose an asynchronous distributed algorithm that implements this original scheme for large decentralized computing networks.

Keywords: distributed optimization

Domains: Machine Learning [stat.ML]

Jerome Fellus, David Picard, Philippe-Henri Gosselin. Asynchronous decentralized convex optimization through short-term gradient averaging. European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Apr 2015, Bruges, Belgium. ⟨hal-01148648⟩