Input Addition and Deletion in Reinforcement: Towards Learning with Structural Changes

Iago Bonnici (1) , Abdelkader Gouaich (1) , Fabien Michel (1)

1 SMILE - Système Multi-agent, Interaction, Langage, Evolution

Reinforcement Learning (RL) agents are commonly thought of as adaptive decision procedures. They work on input/output data streams called łstatesž, łactionsž and łrewardsž. Most current research about RL adaptiveness to changes works under the assumption that the streams signatures (i.e. arity and types of inputs and outputs) remain the same throughout the agent lifetime. As a consequence , natural situations where the signatures vary (e.g. when new data streams become available, or when others become obsolete) are not studied. In this paper, we relax this assumption and consider that signature changes define a new learning situation called Protean Learning (PL). When they occur, traditional RL agents become undefined, so they need to restart learning. Can better methods be developed under the PL view? To investigate this, we first construct a stream-oriented formalism to properly define PL and signature changes. Then, we run experiments in an idealized PL situation where input addition and deletion occur during the learning process. Results show that a simple PL-oriented method enables graceful adaptation of these arity changes, and is more efficient than restarting the process.

Domaines: Intelligence artificielle [cs.AI]

Iago Bonnici, Abdelkader Gouaich, Fabien Michel. Input Addition and Deletion in Reinforcement: Towards Learning with Structural Changes. AAMAS 2020 - 19th International Conference on Autonomous Agents and Multiagent Systems, May 2020, Auckland, New Zealand. pp.177-185, ⟨10.5555/3398761.3398787⟩. ⟨hal-02879819⟩