Incorporating Prior Knowledge in Support Vector Regression

Fabien Lauer (1) , Gérard Bloch (1)

1 CRAN - Centre de Recherche en Automatique de Nancy

This paper explores the addition of constraints to the linear programming formulation of the support vector regression problem for the incorporation of prior knowledge. Equality and inequality constraints are studied with the corresponding types of prior knowledge that can be considered for the method. These include particular points with known values, prior knowledge on any derivative of the function either provided by a prior model or available only at some specific points and bounds on the function or any derivative in a given domain. Moreover, a new method for the simultaneous approximation of multiple outputs linked by some prior knowledge is proposed. This method also allows consideration of different types of prior knowledge on single outputs while training on multiple outputs. Synthetic examples show that incorporating a wide variety of prior knowledge becomes easy, as it leads to linear programs, and helps to improve the approximation in difficult cases. The benefits of the method are finally shown on a real-life application, the estimation of in-cylinder residual gas fraction in spark ignition engines, which is representative of numerous situations met in engineering.

Mots clés: Support Vector Regression; kernel approximation; prior knowledge; multi-outputs; Support Vector Machine

Domaines: Apprentissage [cs.LG]; Autres [stat.ML]

Fabien Lauer, Gérard Bloch. Incorporating Prior Knowledge in Support Vector Regression. Machine Learning, 2008, 70 (1), pp.89-118. ⟨10.1007/s10994-007-5035-5⟩. ⟨hal-00178619⟩