Skip to Main content Skip to Navigation
Conference papers

Explaining Black Boxes on Sequential Data using Weighted Automata

Stéphane Ayache 1, 2 Rémi Eyraud 1, 2, * Noé Goudian 2
* Corresponding author
2 QARMA - éQuipe d'AppRentissage de MArseille
LIS - Laboratoire d'Informatique et Systèmes
Abstract : Understanding how a learned black box works is of crucial interest for the future of Machine Learning. In this paper, we pioneer the question of the global interpretability of learned black box models that assign numerical values to symbolic sequential data. To tackle that task, we propose a spectral algorithm for the extraction of weighted automata (WA) from such black boxes. This algorithm does not require the access to a dataset or to the inner representation of the black box: the inferred model can be obtained solely by querying the black box, feeding it with inputs and analyzing its outputs. Experiments using Recurrent Neural Networks (RNN) trained on a wide collection of 48 synthetic datasets and 2 real datasets show that the obtained approximation is of great quality.
Document type :
Conference papers
Complete list of metadatas

Cited literature [44 references]  Display  Hide  Download
Contributor : Rémi Eyraud <>
Submitted on : Friday, October 5, 2018 - 10:35:51 AM
Last modification on : Monday, December 14, 2020 - 5:28:28 PM
Long-term archiving on: : Sunday, January 6, 2019 - 1:28:00 PM


Files produced by the author(s)


  • HAL Id : hal-01888514, version 1



Stéphane Ayache, Rémi Eyraud, Noé Goudian. Explaining Black Boxes on Sequential Data using Weighted Automata. 14th International Conference on Grammatical Inference, Sep 2018, Wrocław,, Poland. ⟨hal-01888514⟩



Record views


Files downloads