A generic API for load balancing in distributed systems for big data management

Maeva Antoine 1 Laurent Pellegrino 1 Fabrice Huet 1 Françoise Baude 1
1 SCALE - Safe Composition of Autonomous applications with Large-SCALE Execution environment
CRISAM - Inria Sophia Antipolis - Méditerranée , COMRED - COMmunications, Réseaux, systèmes Embarqués et Distribués
Abstract : Distributed systems for big data management very often face the problem of load imbalance among nodes. To address this issue, there exist almost as many load balancing strategies as there are different systems. When designing a scalable distributed system geared towards handling large amounts of information, it is often not so easy to anticipate which kind of strategy will be the most efficient to maintain adequate performance regarding response time, scalability, and reliability at any time. Based on this observation, we describe a generic API to implement and experiment any strategy independently from the rest of the code, prior to a definitive choice for instance. We then show how existing load balancing strategies used by famous systems could be implemented with this API. We also present how this work has helped us implement load balancing on our distributed system and modify the behavior of our strategy in a few lines of code. This led us to easily perform various experiments to determine the most efficient scheme for our system. This paper is an extension to our work presented at Workshop on Parallel and Distributed Computing for Big Data Applications (WPBA) 2014. We detail here more experiments and extend the use of the API to a broad class of big data storage systems
Type de document :
Article dans une revue
Concurrency and Computation: Practice and Experience, Wiley, 2015, <10.1002/cpe.3646>
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01273083
Contributeur : Francoise Baude <>
Soumis le : jeudi 11 février 2016 - 17:51:13
Dernière modification le : vendredi 12 février 2016 - 01:07:55

Identifiants

Collections

Citation

Maeva Antoine, Laurent Pellegrino, Fabrice Huet, Françoise Baude. A generic API for load balancing in distributed systems for big data management. Concurrency and Computation: Practice and Experience, Wiley, 2015, <10.1002/cpe.3646>. <hal-01273083>

Partager

Métriques

Consultations de la notice

171