Hybridizing Constraint Programming and Monte-Carlo Tree Search: Application to the Job Shop problem

Manuel Loth (1) , Michèle Sebag (2) , Youssef Hamadi (1, 3, 4) , Marc Schoenauer (2, 5) , Christian Schulte (6)

1 MSR - INRIA - Microsoft Research - Inria Joint Centre
2 LRI - Laboratoire de Recherche en Informatique
3 LIX - Laboratoire d'informatique de l'École polytechnique [Palaiseau]
4 Microsoft - Microsoft Research [Cambridge]
5 TAO - Machine Learning and Optimisation
6 KTH - KTH Royal Institute of Technology [Stockholm]

Constraint Programming (CP) solvers classically explore the solution space using tree search-based heuristics. Monte-Carlo Tree-Search (MCTS), a tree-search based method aimed at sequential decision making under uncertainty, simultaneously estimates the reward associated to the sub-trees, and gradually biases the exploration toward the most promising regions. This paper examines the tight combination of MCTS and CP on the job shop problem (JSP). The contribution is twofold. Firstly, a reward function compliant with the CP setting is proposed. Secondly, a biased MCTS node-selection rule based on this reward is proposed, that is suitable in a multiple-restarts context. Its integration within the Gecode constraint solver is shown to compete with JSP-specific CP approaches on difficult JSP instances.

Domaines: Apprentissage [cs.LG]; Intelligence artificielle [cs.AI]

Manuel Loth, Michèle Sebag, Youssef Hamadi, Marc Schoenauer, Christian Schulte. Hybridizing Constraint Programming and Monte-Carlo Tree Search: Application to the Job Shop problem. LION7 - Learning and Intelligent OptimizatioN Conference, Jan 2013, Catania, Italy. pp.315-320. ⟨hal-00863453⟩