The Sigma Data Processing Architecture: Leveraging Future Data for Extreme-Scale Data Analytics to Enable High-Precision Decisions

Gabriel Antoniu (1) , Alexandru Costan (1) , Maria S Pérez (2) , Nenad Stojanovic (3)

1 KerData - Scalable Storage for Clouds and Beyond
2 UPM - Universidad Politécnica de Madrid
3 Nissatech

This white paper introduces several key principles based on which HPC-Big Data convergence can be achieved: 1) use future (simulated) data to substantially enrich knowledge obtained based on historical (past) data; 2) enable high-precision analytics thanks to hybrid modeling combining simulation and data-driven models; 3) enable unified data processing thanks to a data processing framework able to relevantly leverage and combine stream processing and batch processing in situ and in transit.

Domaines: Calcul parallèle, distribué et partagé [cs.DC]; Système d'exploitation [cs.OS]; Web

Gabriel Antoniu, Alexandru Costan, Maria S Pérez, Nenad Stojanovic. The Sigma Data Processing Architecture: Leveraging Future Data for Extreme-Scale Data Analytics to Enable High-Precision Decisions. BDEC2 Indiana: Big Data and Extreme-scale Computing, Indiana University Bloomington, Nov 2018, Bloomington, Indiana, United States. pp.3. ⟨hal-03360835⟩