A Diary Study-Based Evaluation Framework for Mobile Information Retrieval

Ourdia Bouidghaghen (1) , Lynda Tamine (1, 2) , Mohand Boughanem (1, 2)

1 IRIT-SIG - Systèmes d’Informations Généralisées
2 UT3 - Université Toulouse III - Paul Sabatier

In this poster, we propose an evaluation framework that investigates the integration of the user context (interests, location and time) into the evaluation process of mobile IR. Our approach is based on a diary study where users are asked to log their queries annotated by their location and time. Users' interests are explicitly acquired or implicitly learned based on users' relevance judgments for the retrieved documents answering their queries. We propose two evaluation protocols namely training/test in chronological order and k-fold cross validation. We exploit this framework in order to evaluate the performance of our context-based personalized mobile search approach. Experimental results show the stability performance of our approach according to the proposed evaluation protocols and demonstrate the viability of the diary approach as a means to capture context in evaluation.

Mots clés: diary study; location; time; user's interests; evaluation framework in mobile context; Experimental evaluation

Domaines: Recherche d'information [cs.IR]

Ourdia Bouidghaghen, Lynda Tamine, Mohand Boughanem. A Diary Study-Based Evaluation Framework for Mobile Information Retrieval. 6th Asia Information Retrieval Societies Conference (AIRS 2010), Dec 2010, Taipei, Taiwan. pp.389-398, ⟨10.1007/978-3-642-17187-1_38⟩. ⟨hal-00553756⟩