Building OLAP cubes on a Cloud Computing environment with MapReduce - Benchmarking Hive
Résumé
Large-scale data analysis has become increasingly important for many enterprises, and Cloud Computing, under the impulse of large companies, has recently endowed a special attention both in industry and academic researches. Hadoop, based on a new distributed computing paradigm, called MapReduce, has allowed to facilitate access to such environments, due to its impressive scalability and flexibility to handle structured as well as unstructured data. The goal of our work is to develop a Cloud Computing environment for exploiting data warehouses and perform online analysis. It consists of handling large nonrelational databases and supporting data warehouse with a new generation of database management systems (DBMS) such as Hive. Thus, to set up such an environment, we implemented a data warehouse under Hadoop and Hive and we used the Map and Reduce functions of this environment, then we compared the cost of loading the warehoused data and constructing OLAP cubes between a virtual and a physical cluster, as well as the rise in data loading on a physical cluster. Obtained results allows MapReduce developers to fully compare the performance, help in the choice of platform, in which a customer application can be developed to translate SQL requests to HQL (Hive-QL) requests, and check if a not-relational model is adequate or not.