Toward Real-time, Many-Task Applications on Large Distributed Systems

Sangho Yi (1) , Derrick Kondo (1) , David P. Anderson (2)

1 MESCAL - Middleware efficiently scalable
2 SSL - Space Sciences Laboratory [Berkeley]

In the age of Grid, Cloud, volunteer computing, massively parallel applications are deployed over tens or hundreds of thousands of resources over short periods of times to complete immense computations. In this work, we consider the problem of deploying such applications with stringent real-time requirements. One major challenge is the server-side management of these tasks, which often number in tens or hundreds of thousands on a centralized server. In this work, we design and implement a real-time task management system for many-task computing, called RT-BOINC. The system gives low O(1) worst-case execution time for task management operations, such as task scheduling, state transitioning, and validation. We implement this system on top of BOINC, a common middleware for volunteer computing. Using micro and macro-benchmarks executed in emulation experiments, we show that RT-BOINC provides significantly lower worst-case execution time, and lessens the gap between the average and the worst-case performance compared with the original BOINC implementation.

Mots clés: Real-time; Volunteer Computing; Grid Computing; Desktop Grid

Domaines: Calcul parallèle, distribué et partagé [cs.DC]; Performance et fiabilité [cs.PF]

Sangho Yi, Derrick Kondo, David P. Anderson. Toward Real-time, Many-Task Applications on Large Distributed Systems. 2010. ⟨hal-00458061⟩