Exploring Large Feature Spaces with Hierarchical Multiple Kernel Learning

Francis Bach 1
1 WILLOW - Models of visual object recognition and scene understanding
DI-ENS - Département d'informatique de l'École normale supérieure, ENS Paris - École normale supérieure - Paris, Inria Paris-Rocquencourt, CNRS - Centre National de la Recherche Scientifique : UMR8548
Abstract : For supervised and unsupervised learning, positive definite kernels allow to use large and potentially infinite dimensional feature spaces with a computational cost that only depends on the number of observations. This is usually done through the penalization of predictor functions by Euclidean or Hilbertian norms. In this paper, we explore penalizing by sparsity-inducing norms such as the l1-norm or the block l1-norm. We assume that the kernel decomposes into a large sum of individual basis kernels which can be embedded in a directed acyclic graph; we show that it is then possible to perform kernel selection through a hierarchical multiple kernel learning framework, in polynomial time in the number of selected kernels. This framework is naturally applied to non linear variable selection; our extensive simulations on synthetic datasets and datasets from the UCI repository show that efficiently exploring the large feature space through sparsity-inducing norms leads to state-of-the-art predictive performance.
Type de document :
Pré-publication, Document de travail
2008
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-00319660
Contributeur : Francis Bach <>
Soumis le : mardi 9 septembre 2008 - 08:44:32
Dernière modification le : jeudi 29 septembre 2016 - 01:22:14
Document(s) archivé(s) le : vendredi 4 juin 2010 - 11:06:43

Fichiers

hkl_hal.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-00319660, version 1
  • ARXIV : 0809.1493

Collections

Citation

Francis Bach. Exploring Large Feature Spaces with Hierarchical Multiple Kernel Learning. 2008. <hal-00319660>

Partager

Métriques

Consultations de
la notice

828

Téléchargements du document

214