Scientific workflows: Past, present and future

Malcolm Atkinson (1) , Sandra Gesing (2) , Johan Montagnat (3) , Ian Taylor (4)

1 School of Informatics [Edimbourg]
2 CRC - Centre for Research Computing
3 Laboratoire I3S - SPARKS - Scalable and Pervasive softwARe and Knowledge Systems
4 School of Computer Sciences & Informatics [Cardiff]

This special issue and our editorial celebrate 10 years of progress with data-intensive or scientific workflows. There have been very substantial advances in the representation of workflows and in the engineering of workflow management systems (WMS). The creation and refinement stages are now well supported, with a significant improvement in usability. Improved abstraction supports cross-fertilisation between different workflow communities and consistent interpretation as WMS evolve. Through such re-engineering the WMS deliver much improved performance, significantly increased scale and sophisticated reliability mechanisms. Further improvement is anticipated from substantial advances in optimisation. We invited papers from those who have delivered these advances and selected 14 to represent today's achievements and representative plans for future progress. This editorial introduces those contributions with an overview and categorisation of the papers. Furthermore, it elucidates responses from a survey of major workflow systems, which provides evidence of substantial progress and a structured index of related papers. We conclude with suggestions on areas where further research and development is needed and offer a vision of future research directions.

Mots clés: Scientific workflows; Scientific method; Optimisation; Performance; Usability

Domaines: Calcul parallèle, distribué et partagé [cs.DC]

Malcolm Atkinson, Sandra Gesing, Johan Montagnat, Ian Taylor. Scientific workflows: Past, present and future. Future Generation Computer Systems, 2017, 75, pp.216 - 227. ⟨10.1016/j.future.2017.05.041⟩. ⟨hal-01544818⟩