Skip to Main content Skip to Navigation
New interface
Conference papers

Sequential Decision-Making under Non-stationary Environments via Sequential Change-point Detection

Emmanuel Hadoux 1 Aurélie Beynier 1 Paul Weng 2 
1 SMA - Systèmes Multi-Agents
LIP6 - Laboratoire d'Informatique de Paris 6
LIP6 - Laboratoire d'Informatique de Paris 6
Abstract : Reinforcement Learning (RL) has been mainly interested in computing an optimal policy for an agent acting in a stationary environment. However, in many real world decision problems the assumption on the stationarity does not hold. One can view a non-stationary environment as a set of contexts (also called modes or modules) where a context corresponds to a possible stationary dynamics of the environment. Even most approaches assume that the number of modes is known, a RL method-Reinforcement Learning with Context Detection (RLCD)-has been recently proposed to learn an a pirori unknown set of contexts and detect context changes. In this paper, we propose a new approach by adapting the tools developed in statistics and more precisely in sequential analysis for detecting an environmental change. Our approach is thus more theoretically founded and necessitates less parameters than RLCD. We also show that our parameters are easier to interpret and therefore easier to tune. Finally, we show experimentally that our approach out-performs the current methods on several application problems.
Complete list of metadata

Cited literature [16 references]  Display  Hide  Download
Contributor : Emmanuel Hadoux Connect in order to contact the contributor
Submitted on : Thursday, September 17, 2015 - 3:43:01 PM
Last modification on : Sunday, June 26, 2022 - 9:48:23 AM
Long-term archiving on: : Tuesday, December 29, 2015 - 7:43:25 AM


Files produced by the author(s)


  • HAL Id : hal-01200817, version 1


Emmanuel Hadoux, Aurélie Beynier, Paul Weng. Sequential Decision-Making under Non-stationary Environments via Sequential Change-point Detection. Learning over Multiple Contexts (LMCE), Sep 2014, Nancy, France. ⟨hal-01200817⟩



Record views


Files downloads