Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Self-Educated Language Agent With Hindsight Experience Replay For Instruction Following

Abstract : Language creates a compact representation of the world and allows the description of unlimited situations and objectives through compositionality. These properties make it a natural fit to guide the training of interactive agents as it could ease recurrent challenges in Reinforcement Learning such as sample complexity, generalization, or multi-tasking. Yet, it remains an open-problem to relate language and RL in even simple instruction following scenarios. Current methods rely on expert demonstrations, auxiliary losses, or inductive biases in neural architectures. In this paper, we propose an orthogonal approach called Textual Hindsight Experience Replay (THER) that extends the Hindsight Experience Replay approach to the language setting. Whenever the agent does not fulfill its instruction, THER learns to output a new directive that matches the agent trajectory, and it relabels the episode with a positive reward. To do so, THER learns to map a state into an instruction by using past successful trajectories, which removes the need to have external expert interventions to relabel episodes as in vanilla HER. We observe that this simple idea also initiates a learning synergy between language acquisition and policy learning on instruction following tasks in the BabyAI environment.
Complete list of metadata
Contributor : Florian Strub Connect in order to contact the contributor
Submitted on : Friday, November 29, 2019 - 1:35:41 PM
Last modification on : Friday, January 21, 2022 - 3:12:34 AM

Links full text


  • HAL Id : hal-02386585, version 1
  • ARXIV : 1910.09451


Geoffrey Cideron, Mathieu Seurin, Florian Strub, Olivier Pietquin. Self-Educated Language Agent With Hindsight Experience Replay For Instruction Following. 2019. ⟨hal-02386585⟩



Les métriques sont temporairement indisponibles