Modeling Non-Uniform Memory Access on Large Compute Nodes with the Cache-Aware Roofline Model

Nicolas Denoyelle (1) , Brice Goglin (1) , Aleksandar Ilic (2) , Emmanuel Jeannot (1) , Leonel Sousa (2)

1 TADAAM - Topology-Aware System-Scale Data Management for High-Performance Computing
2 IST - Instituto Superior Técnico, Universidade Técnica de Lisboa

NUMA platforms, emerging memory architectures with on-package high bandwidth memories bring new opportunities and challenges to bridge the gap between computing power and memory performance. Heterogeneous memory machines feature several performance trade-offs, depending on the kind of memory used, when writing or reading it. Finding memory performance upper-bounds subject to such trade-offs aligns with the numerous interests of measuring computing system performance. In particular, representing applications performance with respect to the platform performance bounds has been addressed in the state-of-the-art Cache-Aware Roofline Model (CARM) to troubleshoot performance issues. In this paper, we present a Locality-Aware extension (LARM) of the CARM to model NUMA platforms bottlenecks, such as contention and remote access. On top of this, the new contribution of this paper is the design and validation of a novel hybrid memory bandwidth model. This new hybrid model quantifies the achievable bandwidth upper-bound under above-described trade-offs with less than 3% error. Hence, when comparing applications performance with the maximum attainable performance, software designers can now rely on more accurate information.

Mots clés: Multi-core/single-chip multiprocessors; Roofline Model; Heterogeneous Memory; Cache-Aware Roofline Model; Skylake; Memory Bandwidth; Shared Memory; Knights Landing; NUMA; Benchmarking; Platform Modeling

Domaines: Architectures Matérielles [cs.AR]; Modélisation et simulation; Calcul parallèle, distribué et partagé [cs.DC]; Système d'exploitation [cs.OS]

Nicolas Denoyelle, Brice Goglin, Aleksandar Ilic, Emmanuel Jeannot, Leonel Sousa. Modeling Non-Uniform Memory Access on Large Compute Nodes with the Cache-Aware Roofline Model. IEEE Transactions on Parallel and Distributed Systems, 2019, 30 (6), pp.1374--1389. ⟨10.1109/TPDS.2018.2883056⟩. ⟨hal-01924951⟩