An Efficient Pipelined Parallel Join Algorithm on Heterogeneous Distributed Architectures

Mohamad Al Hajj Hassan (1) , Mostafa Bamha (1)

1 LIFO - Laboratoire d'Informatique Fondamentale d'Orléans

Pipelined parallelism was largely studied and successfully implemented, on shared nothing machines, in several join algorithms in the presence of ideal conditions of load balancing between processors and in the absence of data skew. The aim of pipelining is to allow flexible resource allocation while avoiding unnecessary disk input/output for intermediate join results in the treatment of multi-join queries. The main drawback of pipelining in existing algorithms is that communication and load balancing remain limited to the use of static approaches (generated during query optimization phase) based on hashing to redistribute data over the network and therefore cannot solve data skew problem and load imbalance between processors on heterogeneous multi-processor architectures where the load of each processor may vary in a dynamic and unpredictable way. In this paper, we present a pipelined parallel algorithm for multi-join queries allowing to solve the problem of data skew while guaranteeing perfect balancing properties, on heterogeneous multi-processor Shared Nothing architectures. The performance of this algorithm is analyzed using the scalable portable BSP (Bulk Synchronous Parallel) cost model.

Domaines: Informatique

Mohamad Al Hajj Hassan, Mostafa Bamha. An Efficient Pipelined Parallel Join Algorithm on Heterogeneous Distributed Architectures. José Cordeiro, Boris Shishkov, AlpeshKumar Ranchordas and Markus Helfert. Software and Data Technologies, Extended and revised ICSFOT'2008 Best papers Book., Springer Berlin Heidelberg, pp.119--133, 2009, Communications in Computer and Information Science, ⟨10.1007/978-3-642-05201-9_10⟩. ⟨hal-00460663⟩