Overview of the interactive task in BioCreative V

Qinghua Wang (1) , Shabbir S. Abdul (2) , Lara Almeida (3) , Sophia Ananiadou (4) , Yalbi I. Balderas-Martinez (5) , Riza Batista-Navarro (4) , David Campos (6) , Lucy Chilton (7) , Hui-Jou Chou (8) , Gabriela Contreras (5) , Laurel Cooper (9) , Hong-Jie Dai (10) , Barbra Ferrell (1) , Juliane Fluck (11) , Socorro Gama-Castro (5) , Nancy George (12) , Georgios Gkoutos (13) , Afroza K. Irin (14) , Lars J. Jensen (15) , Silvia Jimenez (16) , Toni R. Jue (17) , Ingrid Keseler (18) , Sumit Madan (11) , Sergio Matos (3) , Peter Mcquilton (19) , Marija Milacic (20) , Matthew Mort (21) , Jeyakumar Natarajan (22) , Evangelos Pafilis (23) , Emiliano Pereira (24) , Shruti Rao (25) , Fabio Rinaldi (26) , Karen Rothfels (20) , David Salgado (27) , Raquel M. Silva (28) , Onkar Singh (2) , Raymund Stefancsik (29) , Chu-Hsien Su (30) , Suresh Subramani (22) , Hamsa D. Tadepally (31) , Loukia Tsaprouni (32) , Nicole Vasilevsky (33) , Xiaodong Wang (34) , Andrew Chatr-Aryamontri (35) , Stanley J. F. Laulederkind (36) , Sherri Matis-Mitchell (37) , Johanna Mcentyre (38) , Sandra Orchard (38) , Sangya Pundir (38) , Raul Rodriguez-Esteban (39) , Kimberly van Auken (34) , Zhiyong Lu (40) , Mary Schaeffer (41) , Cathy H. Wu (1) , Lynette Hirschman (42) , Cecilia N. Arighi (1)

1 University of Delaware [Newark]
2 Taipei Medical University
3 DETI - Department of Electronics, Telecommunications and Informatics [Aveiro]
4 NaCTeM - National Centre for Text Mining [Manchester]
5 UNAM - Universidad Nacional Autónoma de México = National Autonomous University of Mexico
6 EEB - Ecology and Evolutionary Biology [Tucson]
7 Newcastle University [Newcastle]
8 Rutgers University [Camden]
9 Department of Botany and Plant Pathology
10 Department of Computer Science and Information Engineering
11 Fraunhofer IAIS - Fraunhofer Institute for Intelligent Analysis and Information Systems
12 SourceData - EMBO
13 University of Birmingham [Birmingham]
14 Universität Bonn = University of Bonn
15 University of Copenhagenn
16 EPFL - Ecole Polytechnique Fédérale de Lausanne
17 UNSW - University of New South Wales [Sydney]
18 SRI - SRI International [Menlo Park]
19 University of Oxford
20 OICR - Ontario Institute for Cancer Research [Canada]
21 Institute of Medical Genetics
22 Bharathiar University
23 IMBBC
24 Laboratorio de Organizacion y Evolucion del Genoma
25 GU - Georgetown University [Washington]
26 Institute of Computational Linguistics
27 GMGF - Génétique Médicale et Génomique Fonctionnelle
28 Universidade de Aveiro
29 Genetics, Cambridge University
30 Academia Sinica
31 Freelance scientific curator
32 University of Bedfordshire
33 OHSU - Oregon Health and Science University [Portland]
34 CALTECH - California Institute of Technology
35 UdeM - Université de Montréal
36 University of Wisconsin - Milwaukee
37 Reed Elsevier
38 EMBL-EBI - European Bioinformatics Institute [Hinxton]
39 F. Hoffmann-La Roche [Basel]
40 NCBI - National Center for Biotechnology Information
41 Mizzou - University of Missouri [Columbia]
42 The MITRE corporation

Fully automated text mining (TM) systems promote efficient literature searching, retrieval, and review but are not sufficient to produce ready-to-consume curated documents. These systems are not meant to replace biocurators, but instead to assist them in one or more literature curation steps. To do so, the user interface is an important aspect that needs to be considered for tool adoption. The BioCreative Interactive task (IAT) is a track designed for exploring user-system interactions, promoting development of useful TM tools, and providing a communication channel between the biocuration and the TM communities. In BioCreative V, the IAT track followed a format similar to previous interactive tracks, where the utility and usability of TM tools, as well as the generation of use cases, have been the focal points. The proposed curation tasks are user-centric and formally evaluated by biocurators. In BioCreative V IAT, seven TM systems and 43 biocurators participated. Two levels of user participation were offered to broaden curator involvement and obtain more feedback on usability aspects. The full level participation involved training on the system, curation of a set of documents with and without TM assistance, tracking of time-on-task, and completion of a user survey. The partial level participation was designed to focus on usability aspects of the interface and not the performance per se. In this case, biocurators navigated the system by performing pre-designed tasks and then were asked whether they were able to achieve the task and the level of difficulty in completing the task. In this manuscript, we describe the development of the interactive task, from planning to execution and discuss major findings for the systems tested.

Domaines: Génomique, Transcriptomique et Protéomique [q-bio.GN]

Qinghua Wang, Shabbir S. Abdul, Lara Almeida, Sophia Ananiadou, Yalbi I. Balderas-Martinez, et al.. Overview of the interactive task in BioCreative V. Database - The journal of Biological Databases and Curation, 2016, 2016, ⟨10.1093/database/baw119⟩. ⟨hal-01469079⟩