Flexible and efficient workflow deployement of data-intensive applications on grids with MOTEUR

Tristan Glatard (1, 2) , Johan Montagnat (1) , Diane Lingrand (1) , Xavier Pennec (2)

1 Laboratoire d'Informatique, Signaux, et Systèmes de Sophia-Antipolis (I3S) / Equipe MODALIS
2 ASCLEPIOS - Analysis and Simulation of Biomedical Images

Workflows offer a powerful way to describe and deploy applications on grid infrastructures. Many workflow management systems have been proposed but there is still a lack of a system that would allow both a simple description of the dataflow of the application and an efficient execution on a grid platform. In this paper, we study the requirements of such a system, underlining the need for well-defined data composition strategies on the one hand and for a fully parallel execution on the other. As combining those features is not straightforward, we then propose algorithms to do so and we describe the design and implementation of MOTEUR, a workflow engine that fulfills those requirements. Performance results and overhead quantification are shown to evaluate MOTEUR with respect to existing comparable workflow systems on a production grid.

Domaines: Calcul parallèle, distribué et partagé [cs.DC]

Tristan Glatard, Johan Montagnat, Diane Lingrand, Xavier Pennec. Flexible and efficient workflow deployement of data-intensive applications on grids with MOTEUR. International Journal of High Performance Computing Applications, 2008, 22 (3), pp.347-360. ⟨10.1177/1094342008096067⟩. ⟨hal-00459130⟩