Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

High-Dimensional Non-Linear Variable Selection through Hierarchical Kernel Learning

Francis Bach 1
1 WILLOW - Models of visual object recognition and scene understanding
DI-ENS - Département d'informatique de l'École normale supérieure, Inria Paris-Rocquencourt, CNRS - Centre National de la Recherche Scientifique : UMR8548
Abstract : We consider the problem of high-dimensional non-linear variable selection for supervised learning. Our approach is based on performing linear selection among exponentially many appropriately defined positive definite kernels that characterize non-linear interactions between the original variables. To select efficiently from these many kernels, we use the natural hierarchical structure of the problem to extend the multiple kernel learning framework to kernels that can be embedded in a directed acyclic graph; we show that it is then possible to perform kernel selection through a graph-adapted sparsity-inducing norm, in polynomial time in the number of selected kernels. Moreover, we study the consistency of variable selection in high-dimensional settings, showing that under certain assumptions, our regularization framework allows a number of irrelevant variables which is exponential in the number of observations. Our simulations on synthetic datasets and datasets from the UCI repository show state-of-the-art predictive performance for non-linear regression problems.
Complete list of metadata

Cited literature [79 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-00413473
Contributor : Francis Bach <>
Submitted on : Friday, September 4, 2009 - 11:40:13 AM
Last modification on : Thursday, July 1, 2021 - 5:58:06 PM
Long-term archiving on: : Tuesday, June 15, 2010 - 7:59:32 PM

Files

hkl_hal.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-00413473, version 1
  • ARXIV : 0909.0844

Collections

Citation

Francis Bach. High-Dimensional Non-Linear Variable Selection through Hierarchical Kernel Learning. 2009. ⟨hal-00413473⟩

Share

Metrics

Record views

583

Files downloads

2347