Skip to Main content Skip to Navigation
Conference papers

Sequential Decision-Making under Non-stationary Environments via Sequential Change-point Detection

Emmanuel Hadoux 1 Aurélie Beynier 1 Paul Weng 2
1 SMA - Systèmes Multi-Agents
LIP6 - Laboratoire d'Informatique de Paris 6
2 DECISION
LIP6 - Laboratoire d'Informatique de Paris 6
Abstract : Reinforcement Learning (RL) has been mainly interested in computing an optimal policy for an agent acting in a stationary environment. However, in many real world decision problems the assumption on the stationarity does not hold. One can view a non-stationary environment as a set of contexts (also called modes or modules) where a context corresponds to a possible stationary dynamics of the environment. Even most approaches assume that the number of modes is known, a RL method-Reinforcement Learning with Context Detection (RLCD)-has been recently proposed to learn an a pirori unknown set of contexts and detect context changes. In this paper, we propose a new approach by adapting the tools developed in statistics and more precisely in sequential analysis for detecting an environmental change. Our approach is thus more theoretically founded and necessitates less parameters than RLCD. We also show that our parameters are easier to interpret and therefore easier to tune. Finally, we show experimentally that our approach out-performs the current methods on several application problems.
Complete list of metadatas

Cited literature [16 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01200817
Contributor : Emmanuel Hadoux <>
Submitted on : Thursday, September 17, 2015 - 3:43:01 PM
Last modification on : Thursday, March 21, 2019 - 2:16:43 PM
Long-term archiving on: : Tuesday, December 29, 2015 - 7:43:25 AM

File

LMCE14.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01200817, version 1

Citation

Emmanuel Hadoux, Aurélie Beynier, Paul Weng. Sequential Decision-Making under Non-stationary Environments via Sequential Change-point Detection. Learning over Multiple Contexts (LMCE), Sep 2014, Nancy, France. ⟨hal-01200817⟩

Share

Metrics

Record views

391

Files downloads

736