Improving the Effectiveness of Burst Buffers for Big Data Processing in HPC Systems with Eley

Abstract : Burst Buffer is an effective solution for reducing the data transfer time and the I/O interference in HPC systems. Extending Burst Buffers (BBs) to handle Big Data applications is challenging because BBs must account for the large data inputs of Big Data applications and the Quality-of-Service (QoS) of HPC applications – which are considered as first-class citizens in HPC systems. Existing BBs focus on only intermediate data of Big Data applications and incur a high performance degradation of both Big Data and HPC applications. We present Eley, a burst buffer solution that helps to accelerate the performance of Big Data applications while guaranteeing the QoS of HPC applications. To achieve this goal, Eley embraces interference-aware prefetching technique that makes reading data input faster while introducing low interference for HPC applications. Evaluations using a wide range of Big Data and HPC applications demonstrate that Eley improves the performance of Big Data applications by up to 30% compared to existing BBs while maintaining the QoS of HPC applications.
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01742090
Contributor : Shadi Ibrahim <>
Submitted on : Friday, March 23, 2018 - 5:27:05 PM
Last modification on : Tuesday, March 26, 2019 - 9:25:22 AM

Identifiers

Citation

Orcun Yildiz, Amelie Chi Zhou, Shadi Ibrahim. Improving the Effectiveness of Burst Buffers for Big Data Processing in HPC Systems with Eley. Future Generation Computer Systems, Elsevier, 2018, 86, pp.308-318. 〈10.1016/j.future.2018.03.029〉. 〈hal-01742090〉

Share

Metrics

Record views

446