On the rates of convergence of Parallelized Averaged Stochastic Gradient Algorithms

Antoine Godichon-Baggioni (1) , Sofiane Saadane (2)

1 LMI - Laboratoire de Mathématiques de l'INSA de Rouen Normandie
2 IMT - Institut de Mathématiques de Toulouse UMR5219

The growing interest for high dimensional and functional data analysis led in the last decade to an important research developing a consequent amount of techniques. Parallelized algorithms, which consist in distributing and treat the data into different machines, for example, are a good answer to deal with large samples taking values in high dimensional spaces. We introduce here a parallelized averaged stochastic gradient algorithm, which enables to treat efficiently and recursively the data, and so, without taking care if the distribution of the data into the machines is uniform. The rate of convergence in quadratic mean as well as the asymptotic normality of the parallelized estimates are given, for strongly and locally strongly convex objectives.

Mots clés: Stochastic Gradient Descent; Averaging; Distributed estimation; Central Limit Theorem; Asynchronous parallel optimization

Domaines: Statistiques [stat]

Antoine Godichon-Baggioni, Sofiane Saadane. On the rates of convergence of Parallelized Averaged Stochastic Gradient Algorithms. Statistics, 2020, 54 (3), pp.618-635. ⟨10.1080/02331888.2020.1764557⟩. ⟨hal-01620943⟩