Reward Function Learning for Dialogue Management

Layla El Asri (1) , Romain Laroche (2) , Olivier Pietquin (1)

1 IMS : Information, Multimodalité & Signal
2 Orange Labs [Issy les Moulineaux]

This paper addresses the problem of defining, from data, a reward function in a Reinforcement Learning (RL) problem. This issue is applied to the case of Spoken Dialogue Systems (SDS), which are interfaces enabling users to interact in natural language. A new methodology which, from system evaluation, apportions rewards over the system's state space, is suggested. A corpus of dialogues is collected on-line and then evaluated by experts, assigning a numerical performance score to each dialogue according to the quality of dialogue management. The approach described in this paper infers, from these scores, a locally distributed reward function which can be used on-line. Two algorithms achieving this goal are proposed. These algorithms are tested on an SDS and it is showed that in both cases, the resulting numerical rewards are close to the performance scores and thus, that it is possible to extract relevant information from performance evaluation to optimise on- line learning.

Domaines: Apprentissage [cs.LG]

Layla El Asri, Romain Laroche, Olivier Pietquin. Reward Function Learning for Dialogue Management. STAIRS 2012, Aug 2012, Montpellier, France. pp.95-106, ⟨10.3233/978-1-61499-096-3-95⟩. ⟨hal-00749430⟩