Linear Regression with Random Projections

Odalric Maillard 1 Rémi Munos 1
1 SEQUEL - Sequential Learning
LIFL - Laboratoire d'Informatique Fondamentale de Lille, Inria Lille - Nord Europe, LAGIS - Laboratoire d'Automatique, Génie Informatique et Signal
Abstract : We investigate a method for regression that makes use of a randomly generated subspace $G_P$ (of finite dimension $P$) of a given large (possibly infinite) dimensional function space $F$, for example, $L_{2}([0,1]^d)$. $G_P$ is defined as the span of $P$ random features that are linear combinations of a basis functions of $F$ weighted by random Gaussian i.i.d.~coefficients. We show practical motivation for the use of this approach, detail the link that this random projections method share with RKHS and Gaussian objects theory and prove, both in deterministic and random design, approximation error bounds when searching for the best regression function in $G_P$ rather than in $F$, and derive excess risk bounds for a specific regression algorithm (least squares regression in $G_P$). This paper stresses the motivation to study such methods, thus the analysis developed is kept simple for explanations purpose and leaves room for future developments.
Type de document :
Article dans une revue
Journal of Machine Learning Research, Journal of Machine Learning Research, 2012, 13, pp.2735-2772
Liste complète des métadonnées

Littérature citée [37 références]  Voir  Masquer  Télécharger
Contributeur : Rémi Munos <>
Soumis le : mardi 8 janvier 2013 - 17:21:07
Dernière modification le : jeudi 21 février 2019 - 10:52:49
Document(s) archivé(s) le : mardi 9 avril 2013 - 03:56:45


Fichiers produits par l'(les) auteur(s)


  • HAL Id : hal-00771487, version 1



Odalric Maillard, Rémi Munos. Linear Regression with Random Projections. Journal of Machine Learning Research, Journal of Machine Learning Research, 2012, 13, pp.2735-2772. 〈hal-00771487〉



Consultations de la notice


Téléchargements de fichiers