A Scalability Comparison Study of Data Management Approaches for Smart Metering Systems

Houssem Chihoub (1) , Christine Collet (2)

1 HADAS - Heterogeneous and Adaptive distributed DAta management Systems
2 Grenoble Institute of Technology

—Nowadays, more and more data are being generated and collected in electrical smart grids. Most of these data are coming from smart meters and sensors deployed massively throughout the power grid. As the generation of data is becoming ever more frequent and with the constantly increasing volumes, it is becoming harder and harder to manage and process these data at the scale of a smart grid within legacy systems. In this work, we focus on investigating the scalability and performance of different data management approaches for meter data processing. To this end, we conduct a thorough experimental study of various systems including a parallel relational database system, MapReduce based systems including Hadoop and Spark, and a NoSQL datastore system. Our experiment sets were conducted on up to 140 nodes on Grid5000 and up to 1.4 TB of meter data. Our results demonstrate that parallel relational systems are more suited for most processing types on smart meter data in the smart grid. In contrast, we show that with the appropriate distribution model, data partitioning and modeling choices we achieve very fast and scalable bill computations, the main complex processing for utilities providers.

Domaines: Calcul parallèle, distribué et partagé [cs.DC]; Base de données [cs.DB]

Houssem Chihoub, Christine Collet. A Scalability Comparison Study of Data Management Approaches for Smart Metering Systems. The International Conference on Parallel Processing (ICPP 2016), Aug 2016, Philadelphia, United States. pp.474 - 483, ⟨10.1109/ICPP.2016.61⟩. ⟨hal-01585352⟩