Support vector machines (SVM) in near infrared (NIR) spectroscopy: focus on parameters optimization and model interpretation
Résumé
Support vector machines (SVM) are learning algorithms that present good generalization performance and can model complex non linear boundaries through the use of adapted kernel functions. They have been introduced recently in chemometrics and have proven to be powerful in NIR spectra classification. But one of the major drawbacks of SVM is that training the model requires optimization of the regularization and kernel meta-parameters in order to control the risk of overfitting and the complexity of the boundary. Furthermore the interpretation of the SVM models remains difficult and these tools are then often considered as black box techniques.
We propose a methodological approach to guide the choice of the SVM parameters based on a grid search for minimizing the classification error rate but also relying on the visualization of the number of support vectors (SVs). We also demonstrate the interest of visualizing the SVs in principal components subspaces to go deeper into the interpretation of the trained SVM. The proposed methods are applied on two NIR datasets: the first one is a slightly non linear 2-class problem and the second one a more complex 3-class task. The optimized SVM models are quite parsimonious, relying on 8 and 35 support vectors respectively, and good classification performances is obtained (classification rate of 98.9% and 91% on the test sets, respectively).