Skip to Main content Skip to Navigation
Journal articles

Learning and Reasoning for Cultural Metadata Quality

Anna Bobasheva 1, 2, 3, 4 Fabien Gandon 4 Frédéric Precioso 5 
4 WIMMICS - Web-Instrumented Man-Machine Interactions, Communities and Semantics
CRISAM - Inria Sophia Antipolis - Méditerranée , Laboratoire I3S - SPARKS - Scalable and Pervasive softwARe and Knowledge Systems
5 MAASAI - Modèles et algorithmes pour l’intelligence artificielle
CRISAM - Inria Sophia Antipolis - Méditerranée , UNS - Université Nice Sophia Antipolis (1965 - 2019), JAD - Laboratoire Jean Alexandre Dieudonné, Laboratoire I3S - SPARKS - Scalable and Pervasive softwARe and Knowledge Systems
Abstract : This work combines semantic reasoning and machine learning to create tools that allow curators of the visual art collections to identify and correct the annotations of the artwork as well as to improve the relevance of the content-based search results in these collections. The research is based on the Joconde database maintained by French Ministry of Culture that contains illustrated artwork records from main French public and private museums representing archeological objects, decorative arts, fine arts, historical and scientific documents, etc. The Joconde database includes semantic metadata that describes properties of the artworks and their content. The developed methods create a data pipeline that processes metadata, trains a Convolutional Neural Network image classification model, makes prediction for the entire collection and expands the metadata to be the base for the SPARQL search queries. We developed a set of such queries to identify noise and silence in the human annotations and to search image content with results ranked according to the relevance of the objects quantified by the prediction score provided by the deep learning model. We also developed methods to discover new contextual relationships between the concepts in the metadata by analyzing the contrast between the concepts similarities in the Joconde's semantic model and other vocabularies and we tried to improve the model prediction scores based on the semantic relations. Our results show that cross-fertilization between symbolic AI and machine learning can indeed provide the tools to address the challenges of the museum curators work describing the artwork pieces and searching for the relevant images.
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-03363442
Contributor : Frédéric Precioso Connect in order to contact the contributor
Submitted on : Monday, October 4, 2021 - 11:10:59 AM
Last modification on : Friday, August 5, 2022 - 3:50:54 AM
Long-term archiving on: : Wednesday, January 5, 2022 - 6:21:28 PM

Identifiers

Citation

Anna Bobasheva, Fabien Gandon, Frédéric Precioso. Learning and Reasoning for Cultural Metadata Quality. Journal on Computing and Cultural Heritage, Association for Computing Machinery, 2022, ⟨10.1145/3485844⟩. ⟨hal-03363442⟩

Share

Metrics

Record views

146

Files downloads

81