Learning to Rank System Configurations

Romain Deveaud (1) , Josiane Mothe (1, 2) , Jian-Yun Nie (3)

1 IRIT-SIG - Systèmes d’Informations Généralisées
2 UT2J - Université Toulouse - Jean Jaurès
3 UdeM - Université de Montréal

Information Retrieval (IR) systems heavily rely on a large number of parameters, such as the retrieval model or various query expansion parameters, whose values greatly in uence the overall retrieval effectiveness. However, setting all these parameters individually can often be a tedious task, since they can all affect one another, while also vary for different queries. We propose to tackle this problem by dealing with entire system configurations (i.e. a set of parameters representing an IR system) instead of single parameters, and to apply state-of-the-art Learning to Rank techniques to select the most appropriate configuration for a given query. The experiments we conducted on two TREC AdHoc collections show that this approach is feasible and significantly outperforms the traditional way to configure a system using grid search, as well as the top performing systems of the TREC tracks. We also show an analysis on the impact of different groups of parameters on retrieval effectiveness.

Mots clés: SIGEVI; Retrieval system parameters; Learning to rank

Domaines: Théorie de l'information [cs.IT]; Recherche d'information [cs.IR]

Romain Deveaud, Josiane Mothe, Jian-Yun Nie. Learning to Rank System Configurations. 25th ACM International Conference on Information and Knowledge Management (CIKM 2016), Oct 2016, Indianapolis, United States. pp. 2001-2004. ⟨hal-01682971⟩