Generation of Textual Explanations in XAI: the Case of Semantic Annotation

Jean-Philippe Poli (1) , Wassila Ouerdane (2) , Regis Pierrard (1)

1 LI3A (CEA, LIST) - Intelligence Artificielle et Apprentissage Automatique (CEA, LIST)
2 MICS - Mathématiques et Informatique pour la Complexité et les Systèmes

Semantic image annotation is a field of paramount importance in which deep learning excels. However, some application domains, like security or medicine, may need an explanation of this annotation. Explainable Artificial Intelligence is an answer to this need. In this work, an explanation is a sentence in natural language that is dedicated to human users to provide them clues about the process that leads to the decision: the labels assignment to image parts. We focus on semantic image annotation with fuzzy logic that has proven to be a useful framework that captures both image segmentation imprecision and the vagueness of human spatial knowledge and vocabulary. In this paper, we present an algorithm for textual explanation generation of the semantic annotation of image regions.

Mots clés: artificial intelligence; machine learning; online learning; fuzzy logic; Semantic image annotation; deep learning; Explainable Artificial Intelligence; Trustworthy Artificial intelligence; image segmentation; image retrieval; textual explanation generation; Vocabulary; Annotation; Semantics; Natural language; feature extraction; Explanation; natural language generation; semantic annotation; fuzzy constraint satisfaction problems

Domaines: Intelligence artificielle [cs.AI]; Machine Learning [stat.ML]

Jean-Philippe Poli, Wassila Ouerdane, Regis Pierrard. Generation of Textual Explanations in XAI: the Case of Semantic Annotation. 2021 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Jul 2021, Luxembourg, Luxembourg. pp.9494589, ⟨10.1109/FUZZ45933.2021.9494589⟩. ⟨hal-03408917⟩