Exact Cross-Validation for kNN and applications to passive and active learning in classification

Alain A. Célisse (1) , Tristan Mary-Huard (1)

1 MIA-Paris - Mathématiques et Informatique Appliquées

In the binary classification framework, a closed form expression of the cross-validation Leave-p-Out (LpO) risk estimator for the k Nearest Neighbor algorithm (kNN) is derived. It is first used to study the LpO risk minimization strategy for choosing k in the passive learning setting. The impact of p on the choice of k and the LpO estimation of the risk are inferred. In the active learning setting, a procedure is proposed that selects new examples using a LpO committee of kNN classifiers. The influence of p on the choice of new examples and the tuning of k at each step is investigated. The behavior of k chosen by LpO is shown to be different from what is observed in passive learning.

Mots clés: classification; cross-validation; kNN algorithm; active learning

Domaines: Sciences agricoles

Alain A. Célisse, Tristan Mary-Huard. Exact Cross-Validation for kNN and applications to passive and active learning in classification. Journal de la Société Française de Statistique, 2011, 152 (3), pp.83-97. ⟨hal-01000024⟩