Standardized evaluation of algorithms for computer-aided diagnosis of dementia based on structural MRI: the CADDementia challenge.

Esther E Bron 1 Marion Smits 2, 3 Wiesje M Van Der Flier 4 Hugo Vrenken 4 Frederik Barkhof 4 Philip Scheltens 4 Janne M Papma 2 Rebecca M E Steketee 2 Carolina Méndez Orellana 2 Rozanna Meijboom 2 Madalena Pinto 5 Joana R Meireles 6 Carolina Garrett 7 António J Bastos-Leite 7 Ahmed Abdulkadir 8, 9 Olaf Ronneberger 10, 9 Nicola Amoroso 11, 12 Roberto Bellotti 11, 12 David Cárdenas-Peña 13 Andrés M Álvarez-Meza 13 Chester V Dolph 14 Khan M Iftekharuddin 14 Simon F Eskildsen 15 Pierrick Coupé 16 Vladimir S Fonov 17 Katja Franke 18 Christian Gaser 18 Christian Ledig 19 Ricardo Guerrero 19 Tong Tong 19 Katherine R Gray 19 Elaheh Moradi 20 Jussi Tohka 20 Alexandre Routier 21 Stanley Durrleman 21 Alessia Sarica 22 Giuseppe Di Fatta 23 Francesco Sensi 24 Andrea Chincarini 24 Garry M Smith 25, 23 Zhivko V Stoyanov 23, 25 Lauge Sørensen 26 Mads Nielsen 26 Sabina Tangaro 11 Paolo Inglese 11 Christian Wachinger 27 Martin Reuter 28, 29 John C Van Swieten 2 Wiro J Niessen 30, 2 Stefan Klein 1
21 ARAMIS - Algorithms, models and methods for images and signals of the human brain
Inria Paris-Rocquencourt, UPMC - Université Pierre et Marie Curie - Paris 6, ICM - Institut du Cerveau et de la Moëlle Epinière = Brain and Spine Institute
Abstract : Algorithms for computer-aided diagnosis of dementia based on structural MRI have demonstrated high performance in the literature, but are difficult to compare as different data sets and methodology were used for evaluation. In addition, it is unclear how the algorithms would perform on previously unseen data, and thus, how they would perform in clinical practice when there is no real opportunity to adapt the algorithm to the data at hand. To address these comparability, generalizability and clinical applicability issues, we organized a grand challenge that aimed to objectively compare algorithms based on a clinically representative multi-center data set. Using clinical practice as the starting point, the goal was to reproduce the clinical diagnosis. Therefore, we evaluated algorithms for multi-class classification of three diagnostic groups: patients with probable Alzheimer's disease, patients with mild cognitive impairment and healthy controls. The diagnosis based on clinical criteria was used as reference standard, as it was the best available reference despite its known limitations. For evaluation, a previously unseen test set was used consisting of 354 T1-weighted MRI scans with the diagnoses blinded. Fifteen research teams participated with a total of 29 algorithms. The algorithms were trained on a small training set (n=30) and optionally on data from other sources (e.g., the Alzheimer's Disease Neuroimaging Initiative, the Australian Imaging Biomarkers and Lifestyle flagship study of aging). The best performing algorithm yielded an accuracy of 63.0% and an area under the receiver-operating-characteristic curve (AUC) of 78.8%. In general, the best performances were achieved using feature extraction based on voxel-based morphometry or a combination of features that included volume, cortical thickness, shape and intensity. The challenge is open for new submissions via the web-based framework: http://caddementia.grand-challenge.org.
Type de document :
Article dans une revue
NeuroImage, Elsevier, 2015, 111, pp.562-79. 〈10.1016/j.neuroimage.2015.01.048〉
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01220123
Contributeur : Pierrick Coupé <>
Soumis le : samedi 24 octobre 2015 - 16:06:58
Dernière modification le : mercredi 4 octobre 2017 - 18:00:01

Identifiants

Collections

Citation

Esther E Bron, Marion Smits, Wiesje M Van Der Flier, Hugo Vrenken, Frederik Barkhof, et al.. Standardized evaluation of algorithms for computer-aided diagnosis of dementia based on structural MRI: the CADDementia challenge.. NeuroImage, Elsevier, 2015, 111, pp.562-79. 〈10.1016/j.neuroimage.2015.01.048〉. 〈hal-01220123〉

Partager

Métriques

Consultations de la notice

458