Scaling up Vector Autoregressive Models With Operator-Valued Random Fourier Features

Abstract : A nonparametric approach to Vector Autoregressive Modeling consists in working in vector-valued Reproducing Kernel Hilbert Spaces. The main idea is to build vector-valued models (OKVAR) using operator-valued kernels. Similar to scalar-valued kernels, operator-valued kernels enjoy representer theorems and learning algorithms that heavily depends on training data. We present a new approach to scale up OKVAR models... This contribution aims at scaling up non-linear autoregression models based on operator-valued kernel ($K$) by constructing an explicit feature map function (ORFF) that transforms an input data to a Hilbert space 'embed' in the RKHS induced by $K$. ORFF are constructed in the spirit of Random Fourier Features introduced by Rahimi and Recht. We show that ORFF competes with VAR on stationary linear time-series in terms of time and accuracy. Moreover ORFF is able to compete with OVK accuracy on non-stationary, non-linear time-series (being better than VAR) while keeping low execution time, comparable to VAR.