Feedback Autonomic Provisioning for Guaranteeing Performance in MapReduce Systems

Mihaly Berekmeri 1, 2 Damián Serrano 2 Sara Bouchenak 3 Nicolas Marchand 1 Bogdan Robu 1
1 GIPSA-SYSCO - SYSCO
GIPSA-DA - Département Automatique
3 DRIM - Distribution, Recherche d'Information et Mobilité
LIRIS - Laboratoire d'InfoRmatique en Image et Systèmes d'information
Abstract : Companies have a fast growing amounts of data to process and store, a data explosion is happening next to us. Currentlyone of the most common approaches to treat these vast data quantities are based on the MapReduce parallel programming paradigm.While its use is widespread in the industry, ensuring performance constraints, while at the same time minimizing costs, still providesconsiderable challenges. We propose a coarse grained control theoretical approach, based on techniques that have already provedtheir usefulness in the control community. We introduce the first algorithm to create dynamic models for Big Data MapReduce systems,running a concurrent workload. Furthermore we identify two important control use cases: relaxed performance - minimal resourceand strict performance. For the first case we develop two feedback control mechanism. A classical feedback controller and an evenbasedfeedback, that minimises the number of cluster reconfigurations as well. Moreover, to address strict performance requirements afeedforward predictive controller that efficiently suppresses the effects of large workload size variations is developed. All the controllersare validated online in a benchmark running in a real 60 node MapReduce cluster, using a data intensive Business Intelligenceworkload. Our experiments demonstrate the success of the control strategies employed in assuring service time constraints.
Complete list of metadatas

Cited literature [41 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01297026
Contributor : Nicolas Marchand <>
Submitted on : Thursday, June 2, 2016 - 4:20:58 PM
Last modification on : Thursday, February 7, 2019 - 3:43:19 PM

File

IEEE TCC 2015.pdf
Files produced by the author(s)

Identifiers

Citation

Mihaly Berekmeri, Damián Serrano, Sara Bouchenak, Nicolas Marchand, Bogdan Robu. Feedback Autonomic Provisioning for Guaranteeing Performance in MapReduce Systems. IEEE Transactions on Cloud Computing, 2018, 6 (4), pp.1004-1016. ⟨10.1109/TCC.2016.2550047⟩. ⟨hal-01297026v2⟩

Share

Metrics

Record views

1027

Files downloads

374