Learning Language Models from Images with ReGLL

Leonor Becerra-Bonache (1) , Hendrik Blockeel (2, 3) , Maria Galván (1) , François Jacquenet (1)

1 LHC - Laboratoire Hubert Curien
2 DTAI - Declarative Languages and Artificial Intelligence
3 KU Leuven - Catholic University of Leuven = Katholieke Universiteit Leuven

In this demonstration, we present ReGLL, a system that is able to learn language models taking into account the perceptual context in which the sentences of the model are produced. Thus, ReGLL learns from pairs (Context, Sentence) where: Context is given in the form of an image whose objects have been identified, and Sentence gives a (partial) description of the image. ReGLL uses Inductive Logic Programming Techniques and learns some mappings between n-grams and first order representations of their meanings. The demonstration shows some applications of the language models learned, such as generating relevant sentences describing new images given by the user and translating some sentences from one language to another without the need of any parallel corpus.

Domaines: Apprentissage [cs.LG]; Intelligence artificielle [cs.AI]; Logique en informatique [cs.LO]

Leonor Becerra-Bonache, Hendrik Blockeel, Maria Galván, François Jacquenet. Learning Language Models from Images with ReGLL. ECML/PKDD: European Conference on Machine Learning and Knowledge Discovery in Databases, Sep 2016, Riva del Garda, Italy. pp.55 - 58, ⟨10.1007/978-3-319-46131-1_12⟩. ⟨hal-01386946⟩