Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Connections and Equivalences between the Nyström Method and Sparse Variational Gaussian Processes

Abstract : We investigate the connections between sparse approximation methods for making kernel methods and Gaussian processes (GPs) scalable to massive data, focusing on the Nystr\"om method and the Sparse Variational Gaussian Processes (SVGP). While sparse approximation methods for GPs and kernel methods share some algebraic similarities, the literature lacks a deep understanding of how and why they are related. This is a possible obstacle for the communications between the GP and kernel communities, making it difficult to transfer results from one side to the other. Our motivation is to remove this possible obstacle, by clarifying the connections between the sparse approximations for GPs and kernel methods. In this work, we study the two popular approaches, the Nystr\"om and SVGP approximations, in the context of a regression problem, and establish various connections and equivalences between them. In particular, we provide an RKHS interpretation of the SVGP approximation, and show that the Evidence Lower Bound of the SVGP contains the objective function of the Nystr\"om approximation, revealing the origin of the algebraic equivalence between the two approaches. We also study recently established convergence results for the SVGP and how they are related to the approximation quality of the Nystr\"om method.
Document type :
Preprints, Working Papers, ...
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-03348934
Contributor : Centre de Documentation Eurecom Connect in order to contact the contributor
Submitted on : Monday, September 20, 2021 - 10:22:21 AM
Last modification on : Thursday, November 25, 2021 - 10:44:02 AM

Links full text

Identifiers

  • HAL Id : hal-03348934, version 1
  • ARXIV : 2106.01121

Collections

Citation

Veit Wild, Motonobu Kanagawa, Dino Sejdinovic. Connections and Equivalences between the Nyström Method and Sparse Variational Gaussian Processes. 2021. ⟨hal-03348934⟩

Share

Metrics

Record views

13