Handling limits of high degree vertices in graph processing using MapReduce and Pregel

Abstract : Even if Pregel scales better than MapReduce in graph processing by reducing iteration's disk I/O, while offering an easy programming model using " think like vertex " approach, large scale graph processing is still challenging in the presence of high degree vertices: Communication and load imbalance among processing nodes can have disastrous effects on performance. In this paper, we introduce a scalable MapReduce graph partitioning approach for high degree vertices using a master/slave partitioning allowing to balance communication and computation among processing nodes during all the stages of graph processing. Cost analysis and performance tests of this partitioning are given to show the effectiveness and the scalability of this approach in large scale systems.
Document type :
Reports
Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-01468723
Contributor : Mostafa Bamha <>
Submitted on : Wednesday, February 15, 2017 - 4:45:20 PM
Last modification on : Thursday, February 7, 2019 - 4:46:55 PM
Long-term archiving on : Tuesday, May 16, 2017 - 3:02:45 PM

File

rr2017-02.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01468723, version 1

Collections

Citation

Mohamad Al Hajj Hassan, Mostafa Bamha. Handling limits of high degree vertices in graph processing using MapReduce and Pregel . [Research Report] Université Orléans, INSA Centre Val de Loire, LIFO EA 4022, France. 2017. ⟨hal-01468723⟩

Share

Metrics

Record views

156

Files downloads

183