The world of Independent learners is not Markovian.

Guillaume J. Laurent (1) , Laëtitia Matignon (1) , Nadine Le Fort-Piat (1)

1 FEMTO-ST - Franche-Comté Électronique Mécanique, Thermique et Optique - Sciences et Technologies (UMR 6174)

In multi-agent systems, the presence of learning agents can cause the environment to be non-Markovian from an agent's perspective thus violat- ing the property that traditional single-agent learning methods rely upon. This paper formalizes some known intuition about concurrently learning agents by providing formal conditions that make the environment non- Markovian from an independent (non-communicative) learner's perspec- tive. New concepts are introduced like the divergent learning paths and the observability of the e ects of others' actions. To illustrate the formal concepts, a case study is also presented. These ndings are signi cant because they both help to understand failures and successes of existing learning algorithms as well as being suggestive for future work.

Mots clés: Multi-Agent System; Machine Learning; Reinforcement Learning; Reinforcement Learning.

Domaines: Micro et nanotechnologies/Microélectronique

Guillaume J. Laurent, Laëtitia Matignon, Nadine Le Fort-Piat. The world of Independent learners is not Markovian.. International Journal of Knowledge-Based and Intelligent Engineering Systems, 2011, 15 (1), pp.55-64. ⟨10.3233/KES-2010-0206⟩. ⟨hal-00601941⟩