Model selection for the ℓ2-SVM by following the regularization

Rémi Bonidal (1) , Samy Tindel (2) , Yann Guermeur (1)

1 ABC - Machine Learning and Computational Biology
2 BIGS - Biology, genetics and statistics

For a support vector machine, model selection consists in selecting the kernel function, the values of its parameters, and the amount of regularization. To set the value of the regularization parameter, one can minimize an appropriate objective function over the regularization path. A priori, this requires the availability of two elements: the objective function and an algorithm computing the regularization path at a reduced cost. The literature provides us with several upper bounds and estimates for the leave-one-out cross-validation error of the ℓ2-SVM. However, no algorithm was available so far for fitting the entire regularization path of this machine. In this article, we introduce the first algorithm of this kind. It is involved in the specification of new methods to tune the corresponding penalization coefficient, whose objective function is a leave-one-out error bound or estimate. From a computational point of view, these methods appear especially appropriate when the Gram matrix is of low rank. A comparative study involving state-of-the-art alternatives provides us with an empirical confirmation of this advantage.

Mots clés: regularization path; model selection; ℓ2-SVM; leave-one-out cross-validation error

Domaines: Optimisation et contrôle [math.OC]

Rémi Bonidal, Samy Tindel, Yann Guermeur. Model selection for the ℓ2-SVM by following the regularization. Transactions on Computational Collective Intelligence, 2013, 13, pp.83-112. ⟨10.1007/978-3-642-54455-2_4⟩. ⟨hal-00849720⟩