Workflow fairness control on online and non-clairvoyant distributed computing platforms

Rafael Ferreira da Silva 1 Tristan Glatard 1 Frédéric Desprez 2
1 Images et Modèles
CREATIS - Centre de Recherche en Acquisition et Traitement de l'Image pour la Santé
2 AVALON - Algorithms and Software Architectures for Distributed and HPC Platforms
Inria Grenoble - Rhône-Alpes, LIP - Laboratoire de l'Informatique du Parallélisme
Abstract : Fairly allocating distributed computing resources among workflow executions is critical to multi-user platforms. However, this problem remains mostly studied in clairvoyant and offline conditions, where task durations on resources are known, or the workload and available resources do not vary along time. We consider a non-clairvoyant, online fairness problem where the platform workload, task costs and resource characteristics are unknown and not stationary. We propose a fairness control loop which assigns task priorities based on the fraction of pending work in the workflows. Workflow characteristics and performance on the target resources are estimated progressively, as information becomes available during the execution. Our method is implemented and evaluated on 4 different applications executed in production conditions on the European Grid Infrastructure. Results show that our technique reduces slowdown variability by 3 to 7 compared to first-come-first-served.
Complete list of metadatas

Cited literature [20 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-00849993
Contributor : Béatrice Rayet <>
Submitted on : Friday, August 2, 2013 - 11:35:09 AM
Last modification on : Wednesday, December 12, 2018 - 3:25:27 PM
Long-term archiving on : Wednesday, April 5, 2017 - 7:09:26 PM

File

europar2013-fairness.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-00849993, version 1

Citation

Rafael Ferreira da Silva, Tristan Glatard, Frédéric Desprez. Workflow fairness control on online and non-clairvoyant distributed computing platforms. 19th International Conference Euro-Par 2013, Aug 2013, Aachen, Germany. pp.102-113. ⟨hal-00849993⟩

Share

Metrics

Record views

538

Files downloads

291