Service interruption on Monday 11 July from 12:30 to 13:00: all the sites of the CCSD (HAL, EpiSciences, SciencesConf, AureHAL) will be inaccessible (network hardware connection).
Skip to Main content Skip to Navigation
Conference papers

On the Use of Non-Stationary Policies for Stationary Infinite-Horizon Markov Decision Processes

Bruno Scherrer 1 Boris Lesner 1 
1 MAIA - Autonomous intelligent machine
Inria Nancy - Grand Est, LORIA - AIS - Department of Complex Systems, Artificial Intelligence & Robotics
Abstract : We consider infinite-horizon stationary $\gamma$-discounted Markov Decision Processes, for which it is known that there exists a stationary optimal policy. Using Value and Policy Iteration with some error $\epsilon$ at each iteration, it is well-known that one can compute stationary policies that are $\frac{2\gamma}{(1-\gamma)^2}\epsilon$-optimal. After arguing that this guarantee is tight, we develop variations of Value and Policy Iteration for computing non-stationary policies that can be up to $\frac{2\gamma}{1-\gamma}\epsilon$-optimal, which constitutes a significant improvement in the usual situation when $\gamma$ is close to $1$. Surprisingly, this shows that the problem of ''computing near-optimal non-stationary policies'' is much simpler than that of ''computing near-optimal stationary policies''.
Document type :
Conference papers
Complete list of metadata

Cited literature [27 references]  Display  Hide  Download
Contributor : Bruno Scherrer Connect in order to contact the contributor
Submitted on : Thursday, November 29, 2012 - 1:31:31 PM
Last modification on : Saturday, June 25, 2022 - 7:45:11 PM
Long-term archiving on: : Saturday, December 17, 2016 - 5:48:15 PM


Files produced by the author(s)


  • HAL Id : hal-00758809, version 1
  • ARXIV : 1211.6898



Bruno Scherrer, Boris Lesner. On the Use of Non-Stationary Policies for Stationary Infinite-Horizon Markov Decision Processes. NIPS 2012 - Neural Information Processing Systems, Dec 2012, South Lake Tahoe, United States. ⟨hal-00758809⟩



Record views


Files downloads