Algebraic Markov Decision Processes

Patrice Perny 1 Olivier Spanjaard 1 Paul Weng 1
1 DECISION
LIP6 - Laboratoire d'Informatique de Paris 6
Abstract : In this paper, we provide an algebraic approach to Markov Decision Processes (MDPs), which allows a unified treatment of MDPs and includes many existing models (quantitative or qualitative) as particular cases. In algebraic MDPs, rewards are expressed in a semiring structure, uncertainty is represented by a decomposable plausibility measure valued on a second semiring structure, and preferences over policies are represented by Generalized Expected Utility. We recast the problem of finding an optimal policy at a finite horizon as an algebraic path problem in a decision rule graph where arcs are valued by functions, which justifies the use of the Jacobi algorithm to solve algebraic Bellman equations. In order to show the potential of this general approach, we exhibit new variations of MDPs, admitting complete or partial preference structures, as well as probabilistic or possibilistic representation of uncertainty.
Document type :
Conference papers
Complete list of metadatas

Cited literature [13 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01492606
Contributor : Lip6 Publications <>
Submitted on : Monday, July 10, 2017 - 6:11:02 PM
Last modification on : Thursday, March 21, 2019 - 2:19:28 PM
Long-term archiving on : Wednesday, January 24, 2018 - 6:40:09 PM

File

pub_359_1_1677.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01492606, version 1

Citation

Patrice Perny, Olivier Spanjaard, Paul Weng. Algebraic Markov Decision Processes. 19th International Joint Conference on Artificial Intelligence, Jul 2005, Edinburgh, United Kingdom. pp.1372-1377. ⟨hal-01492606⟩

Share

Metrics

Record views

154

Files downloads

106