Parsimonious Unsupervised and Semi-Supervised Domain Adaptation with Good Similarity Functions

Emilie Morvant (1, 2) , Amaury Habrard (2) , Stéphane Ayache (2, 1)

1 QARMA - éQuipe AppRentissage et MultimediA [Marseille]
2 LHC - Laboratoire Hubert Curien

In this paper, we address the problem of domain adaptation for binary classification. This problem arises when the distributions generating the source learning data and target test data are somewhat different. From a theoretical standpoint, a classifier has better generalization guarantees when the two domain marginal distributions of the input space are close. Classical approaches try mainly to build new projection spaces or to reweight the source data with the objective of moving closer the two distributions. We study an original direction based on a recent framework introduced by Balcan et al. enabling one to learn linear classifiers in an explicit projection space based on a similarity function, not necessarily symmetric nor positive semi-definite. We propose a well founded general method for learning a low-error classifier on target data which is effective with the help of an iterative procedure compatible with Balcan et al.'s framework. A reweighting scheme of the similarity function is then introduced in order to move closer the distri- butions in a new projection space. The hyperparameters and the reweighting quality are controlled by a reverse validation procedure. Our approach is based on a linear programming formulation and shows good adaptation performances with very sparse models. We first consider the challenging unsupervised case where no target label is accessible, which can be helpful when no manual annotation is possible. We also propose a generalization to the semi-supervised case allowing us to consider some few target labels when available. Finally, we evaluate our method on a synthetic problem and on a real image annotation task.

Mots clés: Machine Learning; Transfer Learning; Domain Adaptation; Good Similarity Functions

Domaines: Machine Learning [stat.ML]

Emilie Morvant, Amaury Habrard, Stéphane Ayache. Parsimonious Unsupervised and Semi-Supervised Domain Adaptation with Good Similarity Functions. Knowledge and Information Systems (KAIS), 2012, 33 (2), pp.309-349. ⟨10.1007/s10115-012-0516-7⟩. ⟨hal-00686205⟩