Communication-aware task placement for workflow scheduling on DaaS-based Cloud

Hadrien Croubois (1, 2) , Eddy Caron (1, 2)

1 ENS de Lyon - École normale supérieure de Lyon
2 AVALON - Algorithms and Software Architectures for Distributed and HPC Platforms

Cloud platforms have emerged as a leading solution for computation. In the meantime, large computations have shifted from big parallel tasks to workflows of smaller tasks with data dependencies between them. Task placement is a major issue on Cloud platforms, especially considering the impact of data exchanges on cost and makespan. In this paper, we investigate the consequences of network contention regarding the use of existing scheduling policies on DaaS-based platforms (DaaS for Data as a Service). We show here that the legacy algorithms use inefficient network models. We then modify those algorithms using a new model inspired by DaaS-based Cloud platforms. Thus, we manage to statically pack tasks so that a batch scheduler could deploy many real-time submitted workflows on a dynamic Cloud platform. Simulations of Fork-Join workflows deployment using SimGrid show that our algorithm reduces computation time as well as deployment costs.

Mots clés: Clustering; Workflow; Scheduling; Cloud

Domaines: Calcul parallèle, distribué et partagé [cs.DC]

Hadrien Croubois, Eddy Caron. Communication-aware task placement for workflow scheduling on DaaS-based Cloud. Workshop PDCO 2017. Parallel / Distributed Computing and Optimization, May 2017, Orlando, FL., United States. ⟨hal-01671774⟩