Skip to Main content Skip to Navigation
Conference papers

TýrFS: Increasing Small Files Access Performance with Dynamic Metadata Replication

Abstract : Small files are known to pose major performance challenges for file systems. Yet, such workloads are increasingly common in a number of Big Data Analytics workflows or large-scale HPC simulations. These challenges are mainly caused by the common architecture of most state-of-the-art file systems needing one or multiple metadata requests before being able to read from a file. Small input file size causes the overhead of this metadata management to gain relative importance as the size of each file decreases. In this paper we propose a set of techniques leveraging consistent hashing and dynamic metadata replication to significantly reduce this metadata overhead. We implement such techniques inside a new file system named TýrFS, built as a thin layer above the Týr object store. We prove that TýrFS increases small file access performance up to one order of magnitude compared to other state-of-the-art file systems, while only causing a minimal impact on file write throughput.
Complete list of metadatas

Cited literature [44 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01892691
Contributor : Pierre Matri <>
Submitted on : Wednesday, October 10, 2018 - 6:09:58 PM
Last modification on : Monday, May 4, 2020 - 11:37:38 AM

File

TyrFS___CCGrid_18.pdf
Files produced by the author(s)

Identifiers

Citation

Pierre Matri, María Pérez, Alexandru Costan, Gabriel Antoniu. TýrFS: Increasing Small Files Access Performance with Dynamic Metadata Replication. CCGRID 2018 - 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, May 2018, Washington, United States. pp.452-461, ⟨10.1109/CCGRID.2018.00072⟩. ⟨hal-01892691⟩

Share

Metrics

Record views

306

Files downloads

330