Overview of ExpertLifeCLEF 2018: how far automated identification systems are from the best experts?

Hervé Goëau 1 Pierre Bonnet 2 Alexis Joly 3
2 MI
ICCF - Institut de Chimie de Clermont-Ferrand
3 ZENITH - Scientific Data Management
LIRMM - Laboratoire d'Informatique de Robotique et de Microélectronique de Montpellier, CRISAM - Inria Sophia Antipolis - Méditerranée
Abstract : Automated identification of plants and animals has improved considerably in the last few years, in particular thanks to the recent advances in deep learning. The next big question is how far such automated systems are from the human expertise. Indeed, even the best experts are sometimes confused and/or disagree between each others when validating visual or audio observations of living organism. A picture actually contains only a partial information that is usually not sufficient to determine the right species with certainty. Quantifying this uncertainty and comparing it to the performance of automated systems is of high interest for both computer scientists and expert naturalists. The LifeCLEF 2018 ExpertCLEF challenge presented in this paper was designed to allow this comparison between human experts and automated systems. In total, 19 deep-learning systems implemented by 4 different research teams were evaluated with regard to 9 expert botanists of the French flora. The main outcome of this work is that the performance of state-of-the-art deep learning models is now close to the most advanced human expertise. This paper presents more precisely the resources and assessments of the challenge, summarizes the approaches and systems employed by the participating research groups, and provides an analysis of the main outcomes.
Document type :
Conference papers
Complete list of metadatas

Cited literature [10 references]  Display  Hide  Download

Contributor : Alexis Joly <>
Submitted on : Tuesday, November 6, 2018 - 10:22:39 AM
Last modification on : Wednesday, December 18, 2019 - 2:54:34 PM
Long-term archiving on: Thursday, February 7, 2019 - 1:21:11 PM


Files produced by the author(s)


  • HAL Id : hal-01913244, version 1


Hervé Goëau, Pierre Bonnet, Alexis Joly. Overview of ExpertLifeCLEF 2018: how far automated identification systems are from the best experts?. CLEF: Conference and Labs of the Evaluation Forum, Sep 2018, Avignon, France. ⟨hal-01913244⟩



Record views


Files downloads