Skip to Main content Skip to Navigation
Conference papers

On finding compromise solutions in multiobjective Markov decision processes

Patrice Perny 1 Paul Weng 1
1 DECISION
LIP6 - Laboratoire d'Informatique de Paris 6
Abstract : A Markov Decision Process (MDP) is a general model for solving planning problems under uncertainty. It has been extended to multiobjective MDP to address multicriteria or multiagent problems in which the value of a decision must be evaluated according to several viewpoints, sometimes conflicting. Although most of the studies concentrate on the determination of the set of Pareto-optimal policies, we focus here on a more specialized problem that concerns the direct determination of policies achieving well-balanced tradeoffs. We first explain why this problem cannot simply be solved by optimizing a linear combination of criteria. This leads us to use an alternative optimality concept which formalizes the notion of best compromise solution, i.e. a policy yielding an expected-utility vector as close as possible (w.r.t. Tchebycheff norm) to a reference point. We show that this notion of optimality depends on the initial state. Moreover, it appears that the best compromise policy cannot be found by a direct adaptation of value iteration. In addition, we observe that in some (if not most) situations, the optimal solution can only be obtained with a randomized policy. To overcome all these problems, we propose a solution method based on linear programming and give some experimental results.
Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-01291655
Contributor : Lip6 Publications <>
Submitted on : Monday, March 21, 2016 - 5:23:54 PM
Last modification on : Thursday, March 21, 2019 - 1:05:02 PM

Identifiers

Citation

Patrice Perny, Paul Weng. On finding compromise solutions in multiobjective Markov decision processes. European Conference on Artificial Intelligence Multidisciplinary Workshop on Advances in Preference Handling, Aug 2010, Lisbon, Portugal. pp.969-970, ⟨10.3233/978-1-60750-606-5-969⟩. ⟨hal-01291655⟩

Share

Metrics

Record views

105