Skip to Main content Skip to Navigation
Conference papers

Salamander: a Holistic Scheduling of MapReduce Jobs on Ephemeral Cloud Resources

Mohamed Handaoui 1, 2 Jean-Emile Dartois 3, 1 Laurent Lemarchand 2 Jalil Boukhobza 1, 2
2 Lab-STICC_UBO_CACS_MOCS
IBNM - Institut Brestois du Numérique et des Mathématiques, Lab-STICC - Laboratoire des sciences et techniques de l'information, de la communication et de la connaissance
3 DiverSe - Diversity-centric Software Engineering
Inria Rennes – Bretagne Atlantique , IRISA-D4 - LANGAGE ET GÉNIE LOGICIEL
Abstract : Most cloud data centers are over-provisioned and underutilized, primarily to handle peak loads and sudden failures. This has motivated many researchers to reclaim the unused resources, which are by nature ephemeral, to run data-intensive applications at a lower cost. Hadoop MapReduce is one of those applications. However, it was designed on the assumption that resources are available as long as users pay for the service. In order to make it possible for Hadoop to run on unused (ephemeral) resources, we have designed a heterogeneity and volatility-aware holistic scheduler consisting of three different components: (1) A MapReduce task and job scheduler that relies on a global vision of resource utilization predictions, (2) a scheduler-based data placement strategy that improves the data locality, and (3) a reactive QoS controller that ensures customers’ service-level agreement (SLA) and minimizes interference between co-located workloads. Our framework makes it possible to take advantage of ephemeral resources efficiently. Indeed, for a given set of jobs, it reduces the overall execution time by up to 47.6% and an average of 18.7% as compared to state-of-the-art strategies.
Document type :
Conference papers
Complete list of metadatas

Cited literature [30 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-02497029
Contributor : Handaoui Mohamed <>
Submitted on : Tuesday, March 3, 2020 - 2:13:23 PM
Last modification on : Monday, March 1, 2021 - 3:21:50 AM
Long-term archiving on: : Thursday, June 4, 2020 - 4:16:02 PM

File

PID6379855.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02497029, version 1

Citation

Mohamed Handaoui, Jean-Emile Dartois, Laurent Lemarchand, Jalil Boukhobza. Salamander: a Holistic Scheduling of MapReduce Jobs on Ephemeral Cloud Resources. CCGRID 2020 - 20th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, Nov 2020, Melbourne, Australia. pp.1-10. ⟨hal-02497029⟩

Share

Metrics

Record views

661

Files downloads

340