A Survey of Pipelined Workflow Scheduling: Models and Algorithms

Anne Benoit (1, 2) , Umit V. Catalyurek (3) , Yves Robert (1, 2) , Erik Saule (3)

1 GRAAL - Algorithms and Scheduling for Distributed Heterogeneous Platforms
2 LIP - Laboratoire de l'Informatique du Parallélisme
3 Department of Biomedical Informatics [Columbus]

A large class of applications need to execute the same workflow on different data sets. Efficient execution of such applications necessitates intelligent distribution of the application components and tasks on a parallel machine, and orchestrating the execution by utilizing task-, data-, pipelined-, and replicated-parallelism. The scheduling problem that encompasses all of these techniques is called pipelined workflow scheduling, and has been widely studied in the last decade. Multiple models and algorithms flourished to tackle various programming paradigms, constraints, machine behaviors or goals. This paper surveys the field by summing up and structuring known results and approaches.

Mots clés: parallel systems; workflow programming; filter-stream programming; scheduling; pipeline; throughput; latency; models; algorithms; distributed systems; parallel systems.

Domaines: Calcul parallèle, distribué et partagé [cs.DC]

Anne Benoit, Umit V. Catalyurek, Yves Robert, Erik Saule. A Survey of Pipelined Workflow Scheduling: Models and Algorithms. 2010. ⟨hal-00521712v1⟩