Offline Scheduling of Map and Reduce Tasks on Hadoop Systems

Aymen Jlassi (1, 2) , Patrick Martineau (1, 2) , Vincent Tkindt (2, 1)

1 LIFAT - Laboratoire d'Informatique Fondamentale et Appliquée de Tours
2 Equipe Ordonnancement et Conduite

MapReduce is a model to manage quantities massive of data. It is based on the distributed and parallel execution of tasks over the cluster of machines. Hadoop is an implementation of MapReduce model, it is used to offer BigData services on the cloud. In this paper, we expose the scheduling problem on Hadoop systems. We focus on the offline-scheduling, expose the problem in a mathematic model and use the time-indexed formulation. We aim consider the maximum of constraints of the MapReduce environment. Solutions for the presented model would be a reference for the on-line Schedules in the case of low and medium instances. Our work is useful in term of the problem definition: constraints are based on observations and take into account resources consumption, data locality, heterogeneous machines and workflow management; this paper defines boundaries references to evaluate the online model.

Mots clés: Big Data; MapReduce Model; Hadoop Scheduling Problem; Time Indexed Formulation

Domaines: Calcul parallèle, distribué et partagé [cs.DC]; Modélisation et simulation; Recherche opérationnelle [math.OC]

Aymen Jlassi, Patrick Martineau, Vincent Tkindt. Offline Scheduling of Map and Reduce Tasks on Hadoop Systems. the 5th International Conference on Cloud Computing and Services Science, , May 2015, Lisbone, Portugal. ⟨10.5220/0005483601780185⟩. ⟨hal-01324994⟩