Handling limits of high degree vertices in graph processing using MapReduce and Pregel

Abstract : Even if Pregel scales better than MapReduce in graph processing by reducing iteration's disk I/O, while offering an easy programming model using " think like vertex " approach, large scale graph processing is still challenging in the presence of high degree vertices: Communication and load imbalance among processing nodes can have disastrous effects on performance. In this paper, we introduce a scalable MapReduce graph partitioning approach for high degree vertices using a master/slave partitioning allowing to balance communication and computation among processing nodes during all the stages of graph processing. Cost analysis and performance tests of this partitioning are given to show the effectiveness and the scalability of this approach in large scale systems.
Type de document :
Rapport
[Research Report] Université Orléans, INSA Centre Val de Loire, LIFO EA 4022, France. 2017
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01468723
Contributeur : Mostafa Bamha <>
Soumis le : mercredi 15 février 2017 - 16:45:20
Dernière modification le : vendredi 27 juillet 2018 - 11:10:02
Document(s) archivé(s) le : mardi 16 mai 2017 - 15:02:45

Fichier

rr2017-02.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01468723, version 1

Collections

Citation

Mohamad Al Hajj Hassan, Mostafa Bamha. Handling limits of high degree vertices in graph processing using MapReduce and Pregel . [Research Report] Université Orléans, INSA Centre Val de Loire, LIFO EA 4022, France. 2017. 〈hal-01468723〉

Partager

Métriques

Consultations de la notice

137

Téléchargements de fichiers

133