Skip to Main content Skip to Navigation
Reports

A Lower Bound for the Commission Times in Replication-Based Distributed Storage Systems

Abstract : Efficient resource utilization becomes a major concern as large-scale computing infrastructures such as supercomputers and clouds keep growing in size. Malleability, the possibility for resource managers to dynamically increase or decrease the amount of resources allocated to a job, is a promising way to save energy and cost. However, state-of-the-art parallel and distributed storage systems have not been designed with malleability in mind. The reason is mainly the supposedly high cost of data transfers required by resizing operations. Nevertheless, as network and storage technologies evolve, old assumptions about potential bottlenecks can be revisited. In this study, we model the duration of the commission operation, for which we obtain theoretical lower bounds. We then consider HDFS as a use case, and we show that our lower bound can be used to evaluate the performance of the commission algorithms. We show that the commission in HDFS can be greatly accelerated. With the highlights provided by our lower bound, we suggest improvements to speed the commission in HDFS.
Complete list of metadata

Cited literature [24 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01817638
Contributor : Nathanaël Cheriere <>
Submitted on : Monday, June 18, 2018 - 3:39:47 PM
Last modification on : Thursday, January 7, 2021 - 4:13:51 PM
Long-term archiving on: : Wednesday, September 26, 2018 - 6:22:29 PM

File

RR-9186.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01817638, version 2

Citation

Nathanaël Cheriere, Matthieu Dorier, Gabriel Antoniu. A Lower Bound for the Commission Times in Replication-Based Distributed Storage Systems. [Research Report] RR-9186, Inria Rennes - Bretagne Atlantique. 2018, pp.1-26. ⟨hal-01817638v2⟩

Share

Metrics

Record views

258

Files downloads

151