Overview of the INEX 2013 Social Book Search Track

Marijn Koolen (1) , Gabriella Kazai (2) , Michael Preminger (3) , Antoine Doucet (4)

1 UvA - University of Amsterdam [Amsterdam] = Universiteit van Amsterdam
2 Microsoft - Microsoft Research [Cambridge]
3 HiOA - Oslo and Akershus University College of Applied Sciences [Oslo]
4 Equipe Hultech - Laboratoire GREYC - UMR6072

The goal of the INEX 2013 Social Book Search Track is to evaluate approaches for supporting users in reading, searching, and navigating collections of books based on book metadata, the full texts of digitised books or associated user-generated content. The investigation is focused around three tasks: 1) the Social Book Search (SBS) task investigates the complex nature of relevance in book search and the role of traditional and user-generated book metadata in retrieval, 2) the Prove It (PI) task evaluates focused retrieval approaches for searching pages in books that can confirm or refute a given factual claim, 3) the Structure Extraction (SE) task evaluates automatic techniques for deriving book structure from OCR text and layout information. Both the SBS and SE tasks have a growing number of active participants, while the PI task is only tackled by a small number of core groups. In the SBS task, we extended last year's investigation into the nature of book suggestions from the LibraryThing forums and how they compare to book relevance judgements. We found further support that such suggestions are a valuable alternative to traditional test collections that are based on top-k pooling and editorial relevance judgements. The PI task added a further relevance criterion that pages should not only confirm or refute a given factual claim, but should also come from an authoritative source that is of the appropriate genre. The relevance assessments have not yet commenced at the time of writing. The SE task has reached a record number of active participants and has, for the first time, witnessed an improvement in the state of the art.

Domaines: Traitement du texte et du document

Marijn Koolen, Gabriella Kazai, Michael Preminger, Antoine Doucet. Overview of the INEX 2013 Social Book Search Track. Information Access Evaluation meets Multilinguality, Multimodality, and Visualization" - Fourth International Conference of the Cross-Language Evaluation Forum, CLEF 2013, Sep 2013, Valencia, Spain. 26 p. ⟨hal-01073644⟩