TýrFS: Increasing Small Files Access Performance with Dynamic Metadata Replication

Abstract : Small files are known to pose major performance challenges for file systems. Yet, such workloads are increasingly common in a number of Big Data Analytics workflows or large-scale HPC simulations. These challenges are mainly caused by the common architecture of most state-of-the-art file systems needing one or multiple metadata requests before being able to read from a file. Small input file size causes the overhead of this metadata management to gain relative importance as the size of each file decreases. In this paper we propose a set of techniques leveraging consistent hashing and dynamic metadata replication to significantly reduce this metadata overhead. We implement such techniques inside a new file system named TýrFS, built as a thin layer above the Týr object store. We prove that TýrFS increases small file access performance up to one order of magnitude compared to other state-of-the-art file systems, while only causing a minimal impact on file write throughput.
Type de document :
Communication dans un congrès
CCGRID 2018 - 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, May 2018, Washington, United States. IEEE, 2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID), pp.452-461, 2018, 〈10.1109/CCGRID.2018.00072〉
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01892691
Contributeur : Pierre Matri <>
Soumis le : mercredi 10 octobre 2018 - 18:09:58
Dernière modification le : vendredi 16 novembre 2018 - 01:28:11

Fichier

TyrFS___CCGrid_18.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

Citation

Pierre Matri, María Pérez, Alexandru Costan, Gabriel Antoniu. TýrFS: Increasing Small Files Access Performance with Dynamic Metadata Replication. CCGRID 2018 - 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, May 2018, Washington, United States. IEEE, 2018 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID), pp.452-461, 2018, 〈10.1109/CCGRID.2018.00072〉. 〈hal-01892691〉

Partager

Métriques

Consultations de la notice

153

Téléchargements de fichiers

71