Empirical comparison of semantic similarity measures for technical question answering

M.N. Boukhatem (1, 2) , Davide Buscaldi (3) , Leo Liberti (1, 2, 4, 5)

1 OptimiX - Optimization at LIX
2 LIX - Laboratoire d'informatique de l'École polytechnique [Palaiseau]
3 LIPN - Laboratoire d'Informatique de Paris-Nord
4 CNRS - Centre National de la Recherche Scientifique
5 X - École polytechnique

We consider the task of looking for the answer to a given user question by means of identifying the most relevant document in a technical knowledge base. We briefly introduce the NLP fields related to this task, then discuss what we think are the most promising methods to accomplish the task. The main aim of the paper is to benchmark the chosen methods on two different knowledge bases (one proprietary, one public). Every document in each KB consists of a title and a text describing a solution to a technical problem. Our tests point out that the best method for the task at hand is the use on Sentence Transformers, a deep learning based method using pre-trained language models.

Domaines: Informatique et langage [cs.CL]; Recherche d'information [cs.IR]

M.N. Boukhatem, Davide Buscaldi, Leo Liberti. Empirical comparison of semantic similarity measures for technical question answering. Advances in Databases and Information Systems (ADBIS22), Sep 2022, Torino, Italy. ⟨10.1007/978-3-031-15743-1_16⟩. ⟨hal-03795996⟩