Skip to Main content Skip to Navigation
Journal articles

Spatial Relation Learning for Explainable Image Classification and Annotation in Critical Applications

Abstract : With the recent successes of black-box models in Artificial Intelligence (AI) and the growing interactions between humans and AIs, explainability issues have risen. In this article, in the context of high-stake applications, we propose an approach for explainable classification and annotation of images. It is based on a transparent model, whose reasoning is accessible and human understandable, and on interpretable fuzzy relations that enable to express the vagueness of natural language. The knowledge about relations is set beforehand by an expert and thus training instances do not need to be annotated. The most relevant relations are extracted using a fuzzy frequent itemset mining algorithm in order to build rules, for classification, and constraints, for annotation. We also present two heuristics that make the process of evaluating relations faster. Since the strengths of our approach are the transparency of the model and the interpretability of the relations, an explanation in natural language can be generated. Supported by experimental results, we show that, given a segmentation of the input, our approach is able to successfully perform the target task and generate explanations that were judged as consistent and convincing by a set of participants.
Complete list of metadata
Contributor : Régis Pierrard Connect in order to contact the contributor
Submitted on : Wednesday, November 10, 2021 - 10:47:18 AM
Last modification on : Thursday, April 21, 2022 - 3:14:45 AM
Long-term archiving on: : Friday, February 11, 2022 - 6:05:49 PM


Files produced by the author(s)


  • HAL Id : hal-03418311, version 1



Régis Pierrard, Jean-Philippe Poli, Céline Hudelot. Spatial Relation Learning for Explainable Image Classification and Annotation in Critical Applications. Artificial Intelligence, Elsevier, 2020. ⟨hal-03418311⟩



Record views


Files downloads