Multi-Objective MDPs with Conditional Lexicographic Reward Preferences

Kyle Hollins (1) , Shlomo Zilberstein (1) , Abdel-Illah Mouaddib (2)

1 UMASS - Department of physics and astronomy
2 Equipe MAD - Laboratoire GREYC - UMR6072

Sequential decision problems that involve multiple objectives are prevalent. Consider for example a driver of a semi-autonomous car who may want to optimize competing objectives such as travel time and the effort associated with manual driving. We introduce a rich model called Lexico-graphic MDP (LMDP) and a corresponding planning algorithm called LVI that generalize previous work by allowing for conditional lexicographic preferences with slack. We analyze the convergence characteristics of LVI and establish its game theoretic properties. The performance of LVI in practice is tested within a realistic benchmark problem in the domain of semi-autonomous driving. Finally, we demonstrate how GPU-based optimization can improve the scalability of LVI and other value iteration algorithms for MDPs.

Domaines: Intelligence artificielle [cs.AI]; Système multi-agents [cs.MA]

Kyle Hollins, Shlomo Zilberstein, Abdel-Illah Mouaddib. Multi-Objective MDPs with Conditional Lexicographic Reward Preferences. Twenty-Ninth Conference on Artificial Intelligence (AAAI), Jan 2015, Austin, United States. pp.3418-3424. ⟨hal-01191876⟩