A Machine Learning based Approach to Evaluating Retrieval Systems

Huyen-Trang Vu 1 Patrick Gallinari 1
1 MALIRE - Machine Learning and Information Retrieval
LIP6 - Laboratoire d'Informatique de Paris 6
Abstract : Test collections are essential to evaluate Information Retrieval (IR) systems. The relevance assessment set has been recognized as the key bottleneck in test collection building, especially on very large sized document collections. This paper addresses the problem of efficiently selecting documents to be included in the assessment set. We will show how machine learning techniques can fit this task. This leads to smaller pools than traditional round robin pooling, thus reduces significantly the manual assessment workload. Experimental results on TREC collections consistently demonstrate the effectiveness of our approach according to different evaluation criteria.
Document type :
Conference papers
Complete list of metadatas

Contributor : Lip6 Publications <>
Submitted on : Friday, August 5, 2016 - 2:28:07 PM
Last modification on : Thursday, March 21, 2019 - 1:10:05 PM

Links full text



Huyen-Trang Vu, Patrick Gallinari. A Machine Learning based Approach to Evaluating Retrieval Systems. Proc. Human Language Technology Conference - HLT-NAACL'06, Jun 2006, New-York, United States. pp.399-406, ⟨10.3115/1220835.1220886⟩. ⟨hal-01352077⟩



Record views