HeatPipe: High Throughput, Low Latency Big Data Heatmap with Spark Streaming

Alexandre Perrot (1) , Romain Bourqui (1, 2) , Nicolas Hanusse (1, 3) , David Auber (1)

1 LaBRI - Laboratoire Bordelais de Recherche en Informatique
2 UB - Université de Bordeaux
3 CNRS - Centre National de la Recherche Scientifique

Heatmap visualization is a well-known type of visual-ization to alleviate the overplot problem of point visualiza-tion. As such, it is well suited to visualize Big Data. In order to tackle the velocity problem of Big Data, one has to leverage streaming computations. Recently, canopy clustering was shown to be well suited for Big Data heatmap visualization. In this article, we present how to design a streaming algorithm to compute canopy clustering using Apache Spark. This result is directly applicable to be included into a lambda architecture.

Mots clés: Big Data; Information Visualization; Heatmap; Lambda Architecture

Domaines: Calcul parallèle, distribué et partagé [cs.DC]

Alexandre Perrot, Romain Bourqui, Nicolas Hanusse, David Auber. HeatPipe: High Throughput, Low Latency Big Data Heatmap with Spark Streaming. IV2017 - 21st International Conference on Information Visualisation, Jul 2017, Londres, United Kingdom. ⟨hal-01516888⟩