Algebraic Markov Decision Processes

Patrice Perny 1 Olivier Spanjaard 1 Paul Weng 1
LIP6 - Laboratoire d'Informatique de Paris 6
Abstract : In this paper, we provide an algebraic approach to Markov Decision Processes (MDPs), which allows a unified treatment of MDPs and includes many existing models (quantitative or qualitative) as particular cases. In algebraic MDPs, rewards are expressed in a semiring structure, uncertainty is represented by a decomposable plausibility measure valued on a second semiring structure, and preferences over policies are represented by Generalized Expected Utility. We recast the problem of finding an optimal policy at a finite horizon as an algebraic path problem in a decision rule graph where arcs are valued by functions, which justifies the use of the Jacobi algorithm to solve algebraic Bellman equations. In order to show the potential of this general approach, we exhibit new variations of MDPs, admitting complete or partial preference structures, as well as probabilistic or possibilistic representation of uncertainty.
Document type :
Conference papers
Complete list of metadatas

Cited literature [13 references]  Display  Hide  Download
Contributor : Lip6 Publications <>
Submitted on : Monday, July 10, 2017 - 6:11:02 PM
Last modification on : Thursday, March 21, 2019 - 2:19:28 PM
Long-term archiving on : Wednesday, January 24, 2018 - 6:40:09 PM


Files produced by the author(s)


  • HAL Id : hal-01492606, version 1


Patrice Perny, Olivier Spanjaard, Paul Weng. Algebraic Markov Decision Processes. 19th International Joint Conference on Artificial Intelligence, Jul 2005, Edinburgh, United Kingdom. pp.1372-1377. ⟨hal-01492606⟩



Record views


Files downloads