\ell_p-norm Multiple Kernel Learning with Low-Rank Kernels

Alain Rakotomamonjy (1) , Sukalpa Chanda (2)

1 LITIS - Laboratoire d'Informatique, de Traitement de l'Information et des Systèmes
2 DCSMT - Department of Computer Science and Media Technology

Kernel-based learning algorithms are well-known to poorly scale to large-scale applications. For such large tasks, a common solution is to use low-rank kernel approximation. While several algorithms and theoretical analyses have already been proposed in the literature, for low-rank Support Vector Machine or low-rank Kernel Ridge Regression, this work addresses the problem of scaling $\ell_p$-norm multiple kernel for large learning tasks using low-rank kernel approximations. Our contributions stand on proposing a novel optimization problem, which takes advantage of the low-rank kernel approximations and on introducing a proximal gradient algorithm for solving two different forms of that problem. We also provide theoretical results on the impact of the low-rank approximations over the kernel combination weights. Experimental evidences show that the proposed approaches scale better than the SMO-MKL algorithm for tasks involving about several hundred thousands of examples.

Mots clés: kernel; multiple kernel learning; nystrom approximation

Domaines: Apprentissage [cs.LG]

Alain Rakotomamonjy, Sukalpa Chanda. \ell_p-norm Multiple Kernel Learning with Low-Rank Kernels. 2014. ⟨hal-00820356v3⟩