Skip to Main content Skip to Navigation
Theses

Apprentissage par renforcement en environnement non stationnaire

Abstract : How should an agent act in the face of uncertainty on the evolution of its environment? In this dissertation, we give a Reinforcement Learning perspective on the resolution of nonstationary problems. The question is seen from three different aspects. First, we study the planning vs. re-planning trade-off of tree search algorithms in stationary Markov Decision Processes. We propose a method to lower the computational requirements of such an algorithm while keeping theoretical guarantees on the performance. Secondly, we study the case of environments evolving gradually over time. This hypothesis is expressed through a mathematical framework called Lipschitz Non-Stationary Markov Decision Processes. We derive a risk averse planning algorithm provably converging to the minimax policy in this setting. Thirdly, we consider abrupt temporal evolution in the setting of lifelong Reinforcement Learning. We propose a non-negative transfer method based on the theoretical study of the optimal Q-function’s Lipschitz continuity with respect to the task space. The approach allows to accelerate learning in new tasks. Overall, this dissertation proposes answers to the question of solving Non-Stationary Markov Decision Processes under three different settings.
Complete list of metadata

Cited literature [135 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/tel-02962985
Contributor : Véronique Soullier <>
Submitted on : Friday, October 9, 2020 - 4:05:13 PM
Last modification on : Tuesday, March 16, 2021 - 3:20:07 PM
Long-term archiving on: : Sunday, January 10, 2021 - 6:54:44 PM

File

DTIS20148.1600269665.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : tel-02962985, version 1

Collections

Citation

Erwan Lecarpentier. Apprentissage par renforcement en environnement non stationnaire. Automatique / Robotique. Institut Supérieur de l'Aéronautique et de l'Espace (ISAE), 2020. Français. ⟨tel-02962985⟩

Share

Metrics

Record views

281

Files downloads

236