Skip to Main content Skip to Navigation
Journal articles

A generic API for load balancing in distributed systems for big data management

Maeva Antoine 1 Laurent Pellegrino 1 Fabrice Huet 1 Françoise Baude 1
1 SCALE - Safe Composition of Autonomous applications with Large-SCALE Execution environment
Laboratoire I3S - COMRED - COMmunications, Réseaux, systèmes Embarqués et Distribués
Abstract : Distributed systems for big data management very often face the problem of load imbalance among nodes. To address this issue, there exist almost as many load balancing strategies as there are different systems. When designing a scalable distributed system geared towards handling large amounts of information, it is often not so easy to anticipate which kind of strategy will be the most efficient to maintain adequate performance regarding response time, scalability, and reliability at any time. Based on this observation, we describe a generic API to implement and experiment any strategy independently from the rest of the code, prior to a definitive choice for instance. We then show how existing load balancing strategies used by famous systems could be implemented with this API. We also present how this work has helped us implement load balancing on our distributed system and modify the behavior of our strategy in a few lines of code. This led us to easily perform various experiments to determine the most efficient scheme for our system. This paper is an extension to our work presented at Workshop on Parallel and Distributed Computing for Big Data Applications (WPBA) 2014. We detail here more experiments and extend the use of the API to a broad class of big data storage systems
Complete list of metadata
Contributor : Francoise Baude <>
Submitted on : Thursday, February 11, 2016 - 5:51:13 PM
Last modification on : Tuesday, January 12, 2021 - 8:44:01 AM

Links full text




Maeva Antoine, Laurent Pellegrino, Fabrice Huet, Françoise Baude. A generic API for load balancing in distributed systems for big data management. Concurrency and Computation: Practice and Experience, Wiley, 2015, ⟨10.1002/cpe.3646⟩. ⟨hal-01273083⟩



Record views