Monte-Carlo Tree Search by Best Arm Identification

Abstract : Recent advances in bandit tools and techniques for sequential learning are steadily enabling new applications and are promising the resolution of a range of challenging related problems. We study the game tree search problem, where the goal is to quickly identify the optimal move in a given game tree by sequentially sampling its stochastic payoffs. We develop new algorithms for trees of arbitrary depth, that operate by summarizing all deeper levels of the tree into confidence intervals at depth one, and applying a best arm identification procedure at the root. We prove new sample complexity guarantees with a refined dependence on the problem instance. We show experimentally that our algorithms outperform existing elimination-based algorithms and match previous special-purpose methods for depth-two trees.
Document type :
Conference papers
Complete list of metadatas

Cited literature [23 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01535907
Contributor : Emilie Kaufmann <>
Submitted on : Sunday, November 5, 2017 - 8:36:37 PM
Last modification on : Friday, March 22, 2019 - 1:34:16 AM
Long-term archiving on : Tuesday, February 6, 2018 - 12:46:23 PM

Files

KK17arXiv.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01535907, version 2
  • ARXIV : 1706.02986

Citation

Emilie Kaufmann, Wouter Koolen. Monte-Carlo Tree Search by Best Arm Identification. NIPS 2017 - 31st Annual Conference on Neural Information Processing Systems, Dec 2017, Long Beach, United States. pp.1-23. ⟨hal-01535907v2⟩

Share

Metrics

Record views

178

Files downloads

218