In Situ Statistical Analysis for Parametric Studies

Théophile Terraz 1 Bruno Raffin 1 Alejandro Ribes 2 Yvan Fournier 2
1 DATAMOVE - Data Aware Large Scale Computing
Inria Grenoble - Rhône-Alpes, LIG - Laboratoire d'Informatique de Grenoble
Abstract : In situ processing proposes to reduce storage needs and I/O traffic by processing results of parallel simulations as soon as they are available in the memory of the compute processes. We focus here on computing in situ statistics on the results of N simulations from a parametric study. The classical approach consists in running various instances of the same simulation with different values of input parameters. Results are then saved to disks and statistics are computed post mortem, leading to very I/O intensive applications. Our solution is to develop Melissa, an in situ library running on staging nodes as a parallel server. When starting, simulations connect to Melissa and send the results of each time step to Melissa as soon as they are available. Melissa implements iterative versions of classical statistical operations, enabling to update results as soon as a new time step from a simulation is available. Once all statistics ar updated, the time step can be discarded. We also discuss two different approaches for scheduling simulation runs: the jobs-in-job and the multi-jobs approaches. Experiments run instances of the Computational Fluid Dynamics Open Source solver Code_Saturne. They confirm that our approach enables one to avoid storing simulation results to disk or in memory.
Complete list of metadatas

Cited literature [15 references]  Display  Hide  Download


https://hal.archives-ouvertes.fr/hal-01383860
Contributor : Bruno Raffin <>
Submitted on : Friday, October 21, 2016 - 11:32:59 AM
Last modification on : Thursday, October 11, 2018 - 8:48:05 AM

Licence


Distributed under a Creative Commons Attribution - NonCommercial 4.0 International License

Identifiers

  • HAL Id : hal-01383860, version 1

Collections

Citation

Théophile Terraz, Bruno Raffin, Alejandro Ribes, Yvan Fournier. In Situ Statistical Analysis for Parametric Studies. In Situ Infrastructures for Enabling Extreme-scale Analysis and Visualization (ISAV2016), Nov 2016, Salt Lake City, United States. ⟨hal-01383860⟩

Share

Metrics

Record views

1782

Files downloads

578