A Lower Bound for the Commission Times in Replication-Based Distributed Storage Systems
Une Borne Inférieure pour les Temps de Commission dans les Systèmes de Stockage Distribués basés sur la Réplication
Résumé
Efficient resource utilization becomes a major concern as large-scale computing infrastructures such as supercomputers and clouds keep growing in size. Malleability, the possibility for resource managers to dynamically increase or decrease the amount of resources allocated to a job, is a promising way to save energy and cost.
However, state-of-the-art parallel and distributed storage systems have not been designed with malleability in mind. The reason is mainly the supposedly high cost of data transfers required by resizing operations. Nevertheless, as network and storage technologies evolve, old assumptions about potential bottlenecks can be revisited.
In this study, we model the duration of the commission operation, for which we obtain theoretical lower bounds. We then consider HDFS as a use case, and we show that our lower bound can be used to evaluate the performance of the commission algorithms. We show that the commission in HDFS can be greatly accelerated. With the highlights provided by our lower bound, we suggest improvements to speed the commission in HDFS.
Dans ce rapport, nous modélisons la durée de l’opération de commission de noeuds dans
un système de fichier distribué basé sur la réplication. Nous en obtenons une borne inférieure. Nous considérons ensuite le cas d’utilisation d’HDFS, et nous montrons que notre borne inférieure peut être utilisé pour évaluer la performance des algorithmes de commission. En particulier, la commission dans HDFS peut être grandement accélérée.
Origine : Fichiers produits par l'(les) auteur(s)
Loading...