A Discussion on Parallelization Schemes for Stochastic Vector Quantization Algorithms

Matthieu Durut (1, 2) , Benoît Patra (2, 3) , Fabrice Rossi (4)

1 LTCI - Laboratoire Traitement et Communication de l'Information
2 Lokad
3 LSTA - Laboratoire de Statistique Théorique et Appliquée
4 SAMM - Statistique, Analyse et Modélisation Multidisciplinaire (SAmos-Marin Mersenne)

This paper studies parallelization schemes for stochastic Vector Quantization algorithms in order to obtain time speed-ups using distributed resources. We show that the most intuitive parallelization scheme does not lead to better performances than the sequential algorithm. Another distributed scheme is therefore introduced which obtains the expected speed-ups. Then, it is improved to fit implementation on distributed architectures where communications are slow and inter-machines synchronization too costly. The schemes are tested with simulated distributed architectures and, for the last one, with Microsoft Windows Azure platform obtaining speed-ups up to $32$ Virtual Machines.

Domaines: Machine Learning [stat.ML]; Apprentissage [cs.LG]; Calcul parallèle, distribué et partagé [cs.DC]

Matthieu Durut, Benoît Patra, Fabrice Rossi. A Discussion on Parallelization Schemes for Stochastic Vector Quantization Algorithms. 20-th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN 2012), Apr 2012, Bruges, Belgium. pp.477-482. ⟨hal-00696072⟩