Policy Iteration Algorithms for DEC-POMDPs with discounted rewards

Jilles Dibangoye (1) , Braim Chaid-Draa (1) , Abdel-Illah Mouaddib (2)

1 ULaval - Université Laval [Québec]
2 Equipe MAD - Laboratoire GREYC - UMR6072

Over the past seven years, researchers have been trying to find algorithms for the decentralized control of multiple agent under uncertainty. Unfortunately, most of the standard methods are unable to scale to real-world-size domains. In this paper, we come up with promising new theoretical insights to build scalable algorithms with provable error bounds. In the light of the new theoretical insights, this research revisits the policy iteration algorithm for the decentralized partially observable Markov decision process (DEC-POMDP). We derive and analyze the first point-based policy iteration algorithmswith provable error bounds. Our experimental results show that we are able to successfully solve all tested DEC-POMDP benchmarks: outperforming standard algorithms, both in solution time and policy quality.

Domaines: Intelligence artificielle [cs.AI]; Modélisation et simulation; Système multi-agents [cs.MA]

Jilles Dibangoye, Braim Chaid-Draa, Abdel-Illah Mouaddib. Policy Iteration Algorithms for DEC-POMDPs with discounted rewards. Proc. AAMAS 2009 Workshop on Multi-agent Sequential Decision-Making in Uncertain Domains (MSDM~2009), 2009, Budapest, Hungary. ⟨hal-00968744⟩