Hysteretic Q-Learning : an algorithm for decentralized reinforcement learning in cooperative multi-agent teams.

Laëtitia Matignon (1) , Guillaume J. Laurent (1) , Nadine Le Fort-Piat (1)

1 LAB - Laboratoire d'automatique de Besançon

Multi-agent systems (MAS) are a field of study of growing interest in a variety of domains such as robotics or distributed controls. The article focuses on decentralized reinforcement learning (RL) in cooperative MAS, where a team of independent learning robot (IL) try to coordinate their individual behavior to reach a coherent joint behavior. We assume that each robot has no information about its teammates'actions. To date, RL approaches for such ILs did not guarantee convergence to the optimal joint policy in scenarios where the coordination is difficult. We report an investigation of existing algorithms for the learning of coordination in cooperative MAS, and suggest a Q-Learning extension for ILs, called Hysteretic Q-Learning. This algorithm does not require any additional communication between robots. Its advantages are showing off and compared to other methods on various applications : bimatrix games, collaborative ball balancing task and pursuit domain.

Domaines: Micro et nanotechnologies/Microélectronique

Laëtitia Matignon, Guillaume J. Laurent, Nadine Le Fort-Piat. Hysteretic Q-Learning : an algorithm for decentralized reinforcement learning in cooperative multi-agent teams.. IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS'07., Oct 2007, San Diego, CA., United States. pp.64-69. ⟨hal-00187279⟩