Handling Pregel's limits in big graphs processing in the presence of high degree vertices

Abstract : Even if specialized distributed graph processing systems such as Pregel scale better than pure MapReduce programs, in graph processing, by reducing disk I/O for iterative algorithms while offering an easy programming model using "think like vertex" paradigm, large scale graphs processing is still challenging in the presence of high degree vertices: Communication and load imbalance among processing nodes can have disastrous effects on performance. In this article, we introduce a scalable MapReduce graph partitioning approach for high degree vertices using master/slave partitioning. This partitioning makes Pregel-like systems, in graph processing, scalable and insensitive to the effects of high degree vertices while guaranteeing perfect balancing properties of communication and computation during all the stages of big graphs processing. A cost model and performance analysis are given to show the effectiveness and the scalability of our graph partitioning approach in large scale systems.
Type de document :
Chapitre d'ouvrage
Applications of Big Data Analytics: Trends, Issues, and Challenges, 2018, 9783319764719. 〈10.1007/978-3-319-76472-6〉
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01802435
Contributeur : Mostafa Bamha <>
Soumis le : mardi 29 mai 2018 - 11:49:10
Dernière modification le : jeudi 17 janvier 2019 - 15:10:02

Identifiants

Collections

Citation

Mohamad Al Hajj Hassan, Mostafa Bamha. Handling Pregel's limits in big graphs processing in the presence of high degree vertices. Applications of Big Data Analytics: Trends, Issues, and Challenges, 2018, 9783319764719. 〈10.1007/978-3-319-76472-6〉. 〈hal-01802435〉

Partager

Métriques

Consultations de la notice

40