I/O Scheduling Service for Multi-Application Clusters

Adrien Lebre 1, * Guillaume Huard 2 Yves Denneulin 1 Przemyslaw Sowa 3
* Auteur correspondant
1 MESCAL - Middleware efficiently scalable
ID-IMAG - Informatique et Distribution, Inria Grenoble - Rhône-Alpes
2 MOAIS - PrograMming and scheduling design fOr Applications in Interactive Simulation
ID-IMAG - Informatique et Distribution, Inria Grenoble - Rhône-Alpes, UJF - Université Joseph Fourier - Grenoble 1
Abstract : Distributed applications, especially the ones being I/O intensive, often access the storage subsystem in a non-sequential way (stride requests). Since such behaviors lower the overall system performance, many applications use parallel I/O libraries such as ROMIO to gather and reorder requests. In the meantime, as cluster usage grows, several applications are of-ten executed concurrently, competing for access to storage subsystems and, thus, potentially canceling optimizations brought by Parallel I/O libraries. The aIOLi project aims at optimizing the I/O accesses within the cluster and providing a simple POSIX API. This article presents an extension of aIOLi to address the issue of disjoint accesses generated by different concurrent applications in a cluster. In such a context, good trade-off has to be assessed between performance, fairness and response time. To achieve this, an I/O scheduling algorithm together with a «requests aggregator» that take into account both application access patterns and global system load, have been designed and merged into aIOLi. This improvement led to the implementation of a new generic framework pluggable into any I/O file system layer. A test composed of two concurrent IOR benchmarks showed improvements on read accesses by a factor ranging from 3.5 to 35 with POSIX calls and from 3.3 to 5 with ROMIO, both reference benchmarks executed on a traditional NFS server without any additional optimizations.
keyword : Parallel I/O HPC NFS
Type de document :
Pré-publication, Document de travail
2006
Liste complète des métadonnées

Littérature citée [26 références]  Voir  Masquer  Télécharger

https://hal.archives-ouvertes.fr/hal-00486899
Contributeur : Adrien Lebre <>
Soumis le : jeudi 27 mai 2010 - 10:39:15
Dernière modification le : mardi 4 décembre 2018 - 09:36:08
Document(s) archivé(s) le : jeudi 16 septembre 2010 - 15:19:40

Fichier

aioli-cluster2006.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-00486899, version 1

Collections

Citation

Adrien Lebre, Guillaume Huard, Yves Denneulin, Przemyslaw Sowa. I/O Scheduling Service for Multi-Application Clusters. 2006. 〈hal-00486899〉

Partager

Métriques

Consultations de la notice

289

Téléchargements de fichiers

219