On the Equivalence between Kernel Quadrature Rules and Random Feature Expansions

Francis Bach 1, 2, *
* Auteur correspondant
2 SIERRA - Statistical Machine Learning and Parsimony
DI-ENS - Département d'informatique de l'École normale supérieure, ENS Paris - École normale supérieure - Paris, Inria Paris-Rocquencourt, CNRS - Centre National de la Recherche Scientifique : UMR8548
Abstract : We show that kernel-based quadrature rules for computing integrals can be seen as a special case of random feature expansions for positive definite kernels, for a particular decomposition that always exists for such kernels. We provide a theoretical analysis of the number of required samples for a given approximation error, leading to both upper and lower bounds that are based solely on the eigenvalues of the associated integral operator and match up to logarithmic terms. In particular, we show that the upper bound may be obtained from independent and identically distributed samples from a specific non-uniform distribution, while the lower bound if valid for any set of points. Applying our results to kernel-based quadrature, while our results are fairly general, we recover known upper and lower bounds for the special cases of Sobolev spaces. Moreover, our results extend to the more general problem of full function approximations (beyond simply computing an integral), with results in L2- and L∞-norm that match known results for special cases. Applying our results to random features, we show an improvement of the number of random features needed to preserve the generalization guarantees for learning with Lipschitz-continuous losses.
Type de document :
Article dans une revue
Journal of Machine Learning Research, Journal of Machine Learning Research, 2017, 18 (21), pp.1-38. 〈http://jmlr.org/papers/v18/15-178.html〉
Liste complète des métadonnées

Littérature citée [64 références]  Voir  Masquer  Télécharger

https://hal.archives-ouvertes.fr/hal-01118276
Contributeur : Francis Bach <>
Soumis le : lundi 9 novembre 2015 - 14:39:15
Dernière modification le : vendredi 25 mai 2018 - 12:02:06
Document(s) archivé(s) le : mercredi 10 février 2016 - 10:33:51

Fichiers

kernelexp_jmlr_revision.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01118276, version 2
  • ARXIV : 1502.06800

Collections

Citation

Francis Bach. On the Equivalence between Kernel Quadrature Rules and Random Feature Expansions. Journal of Machine Learning Research, Journal of Machine Learning Research, 2017, 18 (21), pp.1-38. 〈http://jmlr.org/papers/v18/15-178.html〉. 〈hal-01118276v2〉

Partager

Métriques

Consultations de la notice

442

Téléchargements de fichiers

860