Skip to Main content Skip to Navigation
Journal articles

MRSG - A MapReduce Simulator over SimGrid

Abstract : MapReduce is a parallel programming model to process large datasets, and it was inspired by the Map and Reduce primitives from functional languages. Its first implementation was designed to run on large clusters of homogeneous machines. Though, in the last years, the model was ported to different types of environments, such as desktop grid and volunteer computing. To obtain a good performance in these environments, however, it is necessary to adapt some framework mechanisms, such as scheduling and data distribution algorithms. In this paper we present the MRSG simulator, which reproduces the MapReduce work-flow on top of the SimGrid simulation toolkit, and provides an API to implement and evaluate these new algorithms and policies for MapReduce. To evaluate the simulator, we compared its behavior against a real Hadoop MapReduce deployment. The results show an important similarity between the simulated and real executions.
Complete list of metadatas
Contributor : Pierre Sens <>
Submitted on : Wednesday, January 15, 2014 - 11:16:23 PM
Last modification on : Thursday, March 5, 2020 - 4:49:10 PM



Wagner Kolberg, Pedro de Botelho Marcos, Julio C. S. Anjos, Alexandre K. S. Miyazaki, Claudio R. Geyer, et al.. MRSG - A MapReduce Simulator over SimGrid. Parallel Computing, Elsevier, 2013, 39 (4-5), pp.233--244. ⟨10.1016/j.parco.2013.02.001⟩. ⟨hal-00931855⟩



Record views