Skip to Main content Skip to Navigation
Conference papers

Tuning EASY-Backfilling Queues

Jérôme Lelong 1 Valentin Reis 1, 2 Denis Trystram 2, 3
1 DAO - Données, Apprentissage et Optimisation
LJK - Laboratoire Jean Kuntzmann
2 DATAMOVE - Data Aware Large Scale Computing
Inria Grenoble - Rhône-Alpes, LIG - Laboratoire d'Informatique de Grenoble
Abstract : EASY-Backfilling is a popular scheduling heuristic for allocating jobs in large scale High Performance Computing platforms. While its aggressive reservation mechanism is fast and prevents job starvation, it does not try to optimize any scheduling objective per se. We consider in this work the problem of tuning EASY using queue reordering policies. More precisely, we propose to tune the reordering using a simulation-based methodology. For a given system, we choose the policy in order to minimize the average waiting time. This methodology departs from the First-Come, First-Serve rule and introduces a risk on the maximum values of the waiting time, which we control using a queue thresholding mechanism. This new approach is evaluated through a comprehensive experimental campaign on five production logs. In particular, we show that the behavior of the systems under study is stable enough to learn a heuristic that generalizes in a train/test fashion. Indeed, the average waiting time can be reduced consistently (between 11% to 42% for the logs used) compared to EASY, with almost no increase in maximum waiting times. This work departs from previous learning-based approaches and shows that scheduling heuristics for HPC can be learned directly in a policy space.
Complete list of metadatas

Cited literature [26 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01522459
Contributor : Valentin Reis <>
Submitted on : Monday, May 15, 2017 - 10:23:12 AM
Last modification on : Monday, May 4, 2020 - 11:38:06 AM
Document(s) archivé(s) le : Thursday, August 17, 2017 - 12:20:52 AM

File

paper.pdf
Files produced by the author(s)

Identifiers

Citation

Jérôme Lelong, Valentin Reis, Denis Trystram. Tuning EASY-Backfilling Queues. 21st Workshop on Job Scheduling Strategies for Parallel Processing, May 2017, Orlando, United States. pp.43-61, ⟨10.1007/978-3-319-77398-8_3⟩. ⟨hal-01522459⟩

Share

Metrics

Record views

808

Files downloads

1073