Hedgehog: a Performance-Oriented General Purpose Library for Multi-GPU Systems

Alexandre Bardakoff (1, 2) , Timothy Blattner (1) , Bruno Bachelet (2) , Walid Keyrouz (1) , Loïc Yon (2)

1 NIST - National Institute of Standards and Technology [Gaithersburg]
2 LIMOS - Laboratoire d'Informatique, de Modélisation et d'Optimisation des Systèmes

We present Hedgehog, a general-purpose library for taking advantage of powerful compute nodes, multicore CPUs, and multiple GPUs. The novel aspects of Hedgehog are: (1) its explicit representation of a program as a dataflow graph, (2) its pure dataflow-driven scheduling, (3) its maintenance of a computation's localized state via state managers, and (4) its fine control of memory via memory managers. This dataflow approach results in extremely low overhead for task executions (< 1 microsecond) and no-cost profiling at the task level. This allows us to prototype operations that compare favorably with leading libraries such as cuBLAS-XT.

Domaines: Calcul parallèle, distribué et partagé [cs.DC]; Génie logiciel [cs.SE]

Alexandre Bardakoff, Timothy Blattner, Bruno Bachelet, Walid Keyrouz, Loïc Yon. Hedgehog: a Performance-Oriented General Purpose Library for Multi-GPU Systems. GPU Technology Conference (GTC), NVIDIA, Mar 2020, San Diego, United States. ⟨hal-02981031⟩