Explaining Black Boxes on Sequential Data using Weighted Automata

Stéphane Ayache 1, 2 Rémi Eyraud 1, 2, * Noé Goudian 2
* Corresponding author
2 QARMA - éQuipe d'AppRentissage de MArseille
LIS - Laboratoire d'Informatique et Systèmes
Abstract : Understanding how a learned black box works is of crucial interest for the future of Machine Learning. In this paper, we pioneer the question of the global interpretability of learned black box models that assign numerical values to symbolic sequential data. To tackle that task, we propose a spectral algorithm for the extraction of weighted automata (WA) from such black boxes. This algorithm does not require the access to a dataset or to the inner representation of the black box: the inferred model can be obtained solely by querying the black box, feeding it with inputs and analyzing its outputs. Experiments using Recurrent Neural Networks (RNN) trained on a wide collection of 48 synthetic datasets and 2 real datasets show that the obtained approximation is of great quality.
Document type :
Conference papers
Complete list of metadatas

Cited literature [44 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01888514
Contributor : Rémi Eyraud <>
Submitted on : Friday, October 5, 2018 - 10:35:51 AM
Last modification on : Tuesday, April 16, 2019 - 1:41:20 AM
Long-term archiving on : Sunday, January 6, 2019 - 1:28:00 PM

File

Explaining_black_box_on_sequen...
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01888514, version 1

Collections

Citation

Stéphane Ayache, Rémi Eyraud, Noé Goudian. Explaining Black Boxes on Sequential Data using Weighted Automata. 14th International Conference on Grammatical Inference, Sep 2018, Wrocław,, Poland. ⟨hal-01888514⟩

Share

Metrics

Record views

75

Files downloads

52