Skip to Main content Skip to Navigation
New interface
Conference papers

User Scored Evaluation of Non-Unique Explanations for Relational Graph Convolutional Network Link Prediction on Knowledge Graphs

Nicholas Halliwell 1 Fabien Gandon 1 Freddy Lecue 1, 2 
1 WIMMICS - Web-Instrumented Man-Machine Interactions, Communities and Semantics
CRISAM - Inria Sophia Antipolis - Méditerranée , Laboratoire I3S - SPARKS - Scalable and Pervasive softwARe and Knowledge Systems
Abstract : Relational Graph Convolutional Networks (RGCNs) are commonly used on Knowledge Graphs (KGs) to perform black box link prediction. Several algorithms, or explanation methods, have been proposed to explain their predictions. Evaluating performance of explanation methods for link prediction is difficult without ground truth explanations. Furthermore, there can be multiple explanations for a given prediction in a KG. No dataset exists where observations have multiple ground truth explanations to compare against. Additionally, no standard scoring metrics exist to compare predicted explanations against multiple ground truth explanations. In this paper, we introduce a method, including a dataset (FrenchRoyalty-200k), to benchmark explanation methods on the task of link prediction on KGs, when there are multiple explanations to consider. We conduct a user experiment, where users score each possible ground truth explanation based on their understanding of the explanation. We propose the use of several scoring metrics, using relevance weights derived from user scores for each predicted explanation. Lastly, we benchmark this dataset on state-of-the-art explanation methods for link prediction using the proposed scoring metrics.
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-03402766
Contributor : Nicholas Halliwell Connect in order to contact the contributor
Submitted on : Tuesday, October 26, 2021 - 11:10:02 AM
Last modification on : Tuesday, October 25, 2022 - 4:24:06 PM
Long-term archiving on: : Thursday, January 27, 2022 - 6:41:55 PM

File

K_CAP_2021.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03402766, version 1

Citation

Nicholas Halliwell, Fabien Gandon, Freddy Lecue. User Scored Evaluation of Non-Unique Explanations for Relational Graph Convolutional Network Link Prediction on Knowledge Graphs. International Conference on Knowledge Capture, Dec 2021, Virtual Event, United States. ⟨hal-03402766⟩

Share

Metrics

Record views

178

Files downloads

116