Tradeoff to minimize extra-computations and stopping criterion tests for parallel iterative schemes

Olivier Beaumont (1, 2) , El Mostafa Daoudi (3) , Nicolas Maillard (4) , Pierre Manneback (5) , Jean-Louis Roch (6)

1 LaBRI - Laboratoire Bordelais de Recherche en Informatique
2 SCALAPPLIX - Algorithms and high performance computing for grand challenge applications
3 Faculté des sciences [Oujda]
4 UFRGS - Instituto de Informática da UFRGS
5 UMons - Université de Mons
6 MOAIS - PrograMming and scheduling design fOr Applications in Interactive Simulation

Parallel synchronous iterative algorithms are often penalized by global synchro- nization, due to the cost of stopping tests that are achieved. It is well known that such global synchronizations are expensive for parallel implementations on distrib- uted systems, especially on clusters of processors or computational grids, where the heterogeneity and the number of processors imply a large overhead for this global operation. The aim of this work is to propose a new control technique for the stopping tests, original to the best of our knowledge, which enables to reduce the number of global synchronization near to the optimum, while keeping the number of iterations close to the number performed by standard synchronous algorithm. The main advantage of the proposed technique is that the semantic of the sequential al- gorithm is not modified, so that convergence is preserved and identical outputs are guaranteed. Our method is based on an amortized technique inspired by Floyd's and Brent's algorithms to detect periodicity in a sequence.

Mots clés: parallelism; iteration; early termination

Domaines: Calcul parallèle, distribué et partagé [cs.DC]

Olivier Beaumont, El Mostafa Daoudi, Nicolas Maillard, Pierre Manneback, Jean-Louis Roch. Tradeoff to minimize extra-computations and stopping criterion tests for parallel iterative schemes. [Research Report] 2004, pp.13. ⟨hal-00777293⟩