An implementation of a Distributed Stochastic Gradient Descent for Recommender Systems based on Map-Reduce

Manuel Pozo (1) , Raja Chiky (1)

1 ISEP - Institut Supérieur d'Electronique de Paris

This work presents an implementation of a Distributed Stochastic Gradient Descent (DSGD) for Recommender Systems based on Hadoop/MapReduce. Recommender Systems aim at presenting first the information in which users may be more interested. To do this, they analyse a great volume of data that represent the users preferences (e.g. ratings). Thus, this stirs up the need of load-balancing. DSGD is a Matrix Factorization technique that has demonstrated high accuracy and scalability. In this work we expose this algorithm and modify it to improve its accuracy and adaptability to a hadoop cluster. The experimentation phase uses Movie-Lens datasets. Comparisons with other algorithms are given. Results show the good performance of the implementation.

Mots clés: recommender system; collaborative filter-ing; matrix factorization; gradient descent; parallelization

Domaines: Recherche d'information [cs.IR]; Analyse numérique [cs.NA]; Calcul parallèle, distribué et partagé [cs.DC]; Apprentissage [cs.LG]

Manuel Pozo, Raja Chiky. An implementation of a Distributed Stochastic Gradient Descent for Recommender Systems based on Map-Reduce. INTERNATIONAL WORKSHOP ON COMPUTATIONAL INTELLIGENCE FOR MULTIMEDIA UNDERSTANDING (IWCIM), Oct 2015, Prague, Czech Republic. pp.1-5, ⟨10.1109/IWCIM.2015.7347074⟩. ⟨hal-01314906⟩