A study of FMQ heuristic in cooperative multi-agent games.

Laëtitia Matignon (1) , Guillaume J. Laurent (1) , Nadine Le Fort - Piat (1)

1 FEMTO-ST - Franche-Comté Électronique Mécanique, Thermique et Optique - Sciences et Technologies (UMR 6174)

The article focuses on decentralized reinforcement learning (RL) in cooperative multi-agent games, where a team of independent learning agents (ILs) try to coordinate their individual actions to reach an optimal joint action. Within this framework, some algorithms based on Q-learning are proposed in recent works. Especially, we are interested in Distributed Q-learning which finds optimal policies in deterministic games, and in the Frequency Maximum Q value (FMQ) heuristic which is able in partially stochastic matrix games to distinguish if a poor reward received for the same action are due to either miscoordination or to the noisy reward function. Making this distinction is one of the main difficulties to solve stochastic games. Our objective is to find an algorithm able to switch over the updates according to a detection of the cause of noise. In this paper, a modified version of the FMQ heuristic is proposed which achieves this detection and the update adaptation. Moreover, this modified FMQ version is more robust and very easy to set.

Domaines: Micro et nanotechnologies/Microélectronique

Laëtitia Matignon, Guillaume J. Laurent, Nadine Le Fort - Piat. A study of FMQ heuristic in cooperative multi-agent games.. The 7th International Conference on Autonomous Agents and Multiagent Systems. Workshop 10 : Multi-Agent Sequential Decision Making in Uncertain Multi-Agent Domains, aamas'08., May 2008, Estoril, Portugal. pp.77-91. ⟨hal-00342324⟩