Skip to Main content Skip to Navigation
New interface

Apprentissage par renforcement en environnement non stationnaire

Abstract : How should an agent act in the face of uncertainty on the evolution of its environment? In this dissertation, we give a Reinforcement Learning perspective on the resolution of nonstationary problems. The question is seen from three different aspects. First, we study the planning vs. re-planning trade-off of tree search algorithms in stationary Markov Decision Processes. We propose a method to lower the computational requirements of such an algorithm while keeping theoretical guarantees on the performance. Secondly, we study the case of environments evolving gradually over time. This hypothesis is expressed through a mathematical framework called Lipschitz Non-Stationary Markov Decision Processes. We derive a risk averse planning algorithm provably converging to the minimax policy in this setting. Thirdly, we consider abrupt temporal evolution in the setting of lifelong Reinforcement Learning. We propose a non-negative transfer method based on the theoretical study of the optimal Q-function’s Lipschitz continuity with respect to the task space. The approach allows to accelerate learning in new tasks. Overall, this dissertation proposes answers to the question of solving Non-Stationary Markov Decision Processes under three different settings.
Complete list of metadata

Cited literature [135 references]  Display  Hide  Download
Contributor : Cécile André Connect in order to contact the contributor
Submitted on : Friday, October 9, 2020 - 4:05:13 PM
Last modification on : Wednesday, November 3, 2021 - 3:57:58 AM
Long-term archiving on: : Sunday, January 10, 2021 - 6:54:44 PM


Files produced by the author(s)


  • HAL Id : tel-02962985, version 1



Erwan Lecarpentier. Apprentissage par renforcement en environnement non stationnaire. Automatique / Robotique. Institut Supérieur de l'Aéronautique et de l'Espace (ISAE), 2020. Français. ⟨NNT : ⟩. ⟨tel-02962985⟩



Record views


Files downloads