Continual State Representation Learning for Reinforcement Learning using Generative Replay

Abstract : We consider the problem of building a state representation model in a continual fashion. As the environment changes, the aim is to efficiently compress the sensory state's information without losing past knowledge. The learned features are then fed to a Reinforcement Learning algorithm to learn a policy. We propose to use Variational Auto-Encoders for state representation, and Generative Replay, i.e. the use of generated samples, to maintain past knowledge. We also provide a general and statistically sound method for automatic environment change detection. Our method provides efficient state representation as well as forward transfer, and avoids catastrophic forgetting. The resulting model is capable of incrementally learning information without using past data and with a bounded system size.
Document type :
Conference papers
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01951399
Contributor : Hugo Caselles-Dupré <>
Submitted on : Tuesday, December 11, 2018 - 2:04:50 PM
Last modification on : Thursday, February 7, 2019 - 2:42:27 PM
Document(s) archivé(s) le : Tuesday, March 12, 2019 - 2:49:53 PM

File

_NIPS_CL_Workshop__Continual_S...
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01951399, version 1

Citation

Hugo Caselles-Dupré, Michael Garcia-Ortiz, David Filliat. Continual State Representation Learning for Reinforcement Learning using Generative Replay. Workshop on Continual Learning, NeurIPS 2018 - Thirty-second Conference on Neural Information Processing Systems, Dec 2018, Montréal, Canada. ⟨hal-01951399⟩

Share

Metrics

Record views

40

Files downloads

26