A Polynomial Algorithm for Decentralized Markov Decision Processes with Temporal Constraints

Aurélie Beynier (1) , Abdel-Illah Mouaddib (2)

1 GREYC - Groupe de Recherche en Informatique, Image et Instrumentation de Caen
2 Equipe MAD - Laboratoire GREYC - UMR6072

One of the difficulties to adapt MDPs for the control of cooperative multi-agent systems, is the complexity issued from Decentralized MDPs. Moreover, existing approaches can not be used for real applications because they do not take into account complex constraints about the execution. In this paper, we present a class of DEC-MDPs, OC-DEC-MDP, that can handle temporal and precedence constraints. This model allows several autonomous agents to cooperate so as to complete a set of tasks without communication. In order to allow the agents to coordinate, we introduce an opportunity cost. Each agent builds its own local MDP independently of the other agents but, it takes into account the lost in value provoked, by its local decision, on the other agents. Existing approaches solving DEC-MDP are NEXP complete or exponential, while our OC-DEC-MDP can be solved by a polynomial algorithm with good approximation.

Mots clés: Multi-agent systems; Markov decision Processes; Planning; Uncertainty

Domaines: Intelligence artificielle [cs.AI]; Système multi-agents [cs.MA]

Aurélie Beynier, Abdel-Illah Mouaddib. A Polynomial Algorithm for Decentralized Markov Decision Processes with Temporal Constraints. Fourth International joint Conference on Autonomous Agents and Multi Agent Systems (AAMAS'05), 2005, Utrecht, Netherlands. pp.963-969, ⟨10.1145/1082473.1082619⟩. ⟨hal-01344441⟩