Discovering Causal Structure with Reproducing-Kernel Hilbert Space ε-Machines

Nicolas Brodu (1) , James P Crutchfield (2)

1 GeoStat - Geometry and Statistics in acquisition data
2 UC Davis - University of California [Davis]

We merge computational mechanics' definition of causal states (predictively-equivalent histories) with reproducing-kernel Hilbert space (RKHS) representation inference. The result is a widely applicable method that infers causal structure directly from observations of a system's behaviors whether they are over discrete or continuous events or time. A structural representation – a finite or infinite-state kernel-machine – is extracted by a reduced-dimension transform that gives an efficient representation of causal states and their topology. In this way, the system dynamics are represented by a stochastic (ordinary or partial) differential equation that acts on causal states. We introduce an algorithm to estimate the associated evolution operator. Paralleling the Fokker-Plank equation, it efficiently evolves causal-state distributions and makes predictions in the original data space via an RKHS functional mapping. We demonstrate these techniques, together with their predictive abilities, on discrete-time, discrete-value infinite Markov-order processes generated by finite-state hidden Markov models with (i) finite or (ii) uncountably-infinite causal states and (iii) a continuous-time, continuous-value process generated by a thermally-driven chaotic flow. The method robustly estimates causal structure in the presence of varying external and measurement noise levels.

Mots clés: Stochastic process; Hidden Markov model; Epsilon-machine; Causal states; Hilbert space representation; Data-based modeling; Inference; Predictive model; Time series

Domaines: Systèmes dynamiques [math.DS]; Machine Learning [stat.ML]; Dynamique Chaotique [nlin.CD]; Apprentissage [cs.LG]

Nicolas Brodu, James P Crutchfield. Discovering Causal Structure with Reproducing-Kernel Hilbert Space ε-Machines. 2020. ⟨hal-03029188v1⟩