Comparing System-response Retrieval Models for Open-domain and Casual Conversational Agent

Abstract : This paper studies corpus-based process to select a system-response usable both in chatterbot or as a fallback strategy. It presents, evaluates and compares two selection methods that retrieve and adapt a system-response from the OpenSubtitles2016 corpus given a human-utterance. A corpus of 800 annotated pairs is constituted. Evaluation consists in objective metrics and subjective annotation based on the validity schema proposed in the RE-WOCHAT shared task. Our study indicates that the task of assessing the validity of a system-response given a human-utterance is subjective to an important extent, and is thus a difficult task. Comparisons show that the selection method based on word embedding performs objectively better than the one based on TF-IDF in terms of response variety and response length.
Type de document :
Communication dans un congrès
Second Workshop on Chatbots and Conversational Agent Technologies (WOCHAT@IVA2016), 2016, Los Angeles, United States
Liste complète des métadonnées

Littérature citée [19 références]  Voir  Masquer  Télécharger

https://hal.archives-ouvertes.fr/hal-01782262
Contributeur : Guillaume Dubuisson Duplessis <>
Soumis le : mardi 1 mai 2018 - 14:57:03
Dernière modification le : jeudi 17 mai 2018 - 01:14:26
Document(s) archivé(s) le : lundi 24 septembre 2018 - 18:31:51

Fichier

RP-269.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01782262, version 1

Collections

Citation

Franck Charras, Guillaume Dubuisson Duplessis, Vincent Letard, Anne-Laure Ligozat, Sophie Rosset. Comparing System-response Retrieval Models for Open-domain and Casual Conversational Agent. Second Workshop on Chatbots and Conversational Agent Technologies (WOCHAT@IVA2016), 2016, Los Angeles, United States. 〈hal-01782262〉

Partager

Métriques

Consultations de la notice

36

Téléchargements de fichiers

23