Skip to Main content Skip to Navigation
Conference papers

Resilient Optimistic Termination Detection for the Async-Finish Model

Abstract : Driven by increasing core count and decreasing mean-time-to-failure in supercomputers, HPC runtime systems must improve support for dynamic task-parallel execution and resilience to failures. The async-finish task model, adapted for distributed systems as the asynchronous partitioned global address space programming model, provides a simple way to decompose a computation into nested task groups, each managed by a ‘finish’ that signals the termination of all tasks within the group. For distributed termination detection, maintaining a consistent view of task state across multiple unreliable processes requires additional book-keeping when creating or completing tasks and finish-scopes. Runtime systems which perform this book-keeping pessimistically, i.e. synchronously with task state changes, add a high communication overhead compared to non-resilient protocols. In this paper, we propose optimistic finish, the first message-optimal resilient termination detection protocol for the async-finish model. By avoiding the communication of certain task and finish events, this protocol allows uncertainty about the global structure of the computation which can be resolved correctly at failure time, thereby reducing the overhead for failure-free execution. Performance results using micro-benchmarks and the LULESH hydrodynamics proxy application show significant reductions in resilience overhead with optimistic finish compared to pessimistic finish. Our optimistic finish protocol is applicable to any task-based runtime system offering automatic termination detection for dynamic graphs of non-migratable tasks.
Complete list of metadata

Cited literature [10 references]  Display  Hide  Download
Contributor : Sara Hamouda Connect in order to contact the contributor
Submitted on : Thursday, July 11, 2019 - 1:14:47 PM
Last modification on : Tuesday, March 23, 2021 - 9:28:03 AM


Files produced by the author(s)



Sara Hamouda, Josh Milthorpe. Resilient Optimistic Termination Detection for the Async-Finish Model. 34th International Conference on High Performance Computing, Jun 2019, Frankfurt, Germany. pp.291-311, ⟨10.1007/978-3-030-20656-7_15⟩. ⟨hal-02169496⟩



Record views


Files downloads