Deciding the Value 1 Problem for #-acyclic Partially Observable Markov Decision Processes

Hugo Gimbert (1) , Youssouf Oualhadj (2, 3)

1 LaBRI - Laboratoire Bordelais de Recherche en Informatique
2 LIF - Laboratoire d'informatique Fondamentale de Marseille
3 UMons - Université de Mons

The value 1 problem is a natural decision problem in algorithmic game theory. For partially observable Markov decision processes with reachability objective, this problem is defined as follows: are there strategies that achieve the reachability objective with probability arbitrarily close to 1? This problem was shown undecidable recently. Our contribution is to introduce a class of partially observable Markov decision processes, namely #-acyclic partially observable Markov decision processes, for which the value 1 problem is decidable. Our algorithm is based on the construction of a two-player perfect information game, called the knowledge game, abstracting the behaviour of a #-acyclic partially observable Markov decision process M such that the first player has a winning strategy in the knowledge game if and only if the value of M is 1.

Domaines: Informatique et théorie des jeux [cs.GT]

Hugo Gimbert, Youssouf Oualhadj. Deciding the Value 1 Problem for #-acyclic Partially Observable Markov Decision Processes. 2012. ⟨hal-00743137v3⟩