Optimizing Asynchronous Multi-Level Checkpoint/Restart Configurations with Machine Learning - Archive ouverte HAL Accéder directement au contenu
Communication Dans Un Congrès Année : 2020

Optimizing Asynchronous Multi-Level Checkpoint/Restart Configurations with Machine Learning

Résumé

With the emergence of versatile storage systems, multi-level checkpointing (MLC) has become a common approach to gain efficiency. However, multi-level checkpoint/restart can cause enormous I/O traffic on HPC systems. To use multi-level checkpointing efficiently, it is important to optimize check-point/restart configurations. Current approaches, namely model-ing and simulation, are either inaccurate or slow in determining the optimal configuration for a large scale system. In this paper, we show that machine learning models can be used in combination with accurate simulation to determine the optimal checkpoint configurations. We also demonstrate that more advanced techniques such as neural networks can further improve the performance in optimizing checkpoint configurations.
Fichier principal
Vignette du fichier
09150452.pdf (1.06 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-02914478 , version 1 (11-08-2020)

Identifiants

Citer

Tonmoy Dey, Kento Sato, Bogdan Nicolae, Jian Guo, Jens Domke, et al.. Optimizing Asynchronous Multi-Level Checkpoint/Restart Configurations with Machine Learning. IPDPSW'20: The 2020 IEEE International Parallel and Distributed Processing Symposium Workshops, May 2020, New Orleans, United States. pp.1036-1043, ⟨10.1109/IPDPSW50202.2020.00174⟩. ⟨hal-02914478⟩
67 Consultations
192 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More