A New PAC-Bayesian Perspective on Domain Adaptation

Pascal Germain (1) , Amaury Habrard (2) , François Laviolette (3) , Emilie Morvant (2)

1 SIERRA - Statistical Machine Learning and Parsimony
2 LHC - Laboratoire Hubert Curien
3 Department of Computer Science and Software Engineering [Québec]

We study the issue of PAC-Bayesian domain adaptation: We want to learn, from a source domain, a majority vote model dedicated to a target one. Our theoretical contribution brings a new perspective by deriving an upper-bound on the target risk where the distributions’ divergence—expressed as a ratio—controls the trade-off between a source error measure and the target voters’ disagreement. Our bound suggests that one has to focus on regions where the source data is informative. From this result, we derive a PAC-Bayesian generalization bound, and specialize it to linear classifiers. Then, we infer a learning algorithm and perform experiments on real data.

Mots clés: Statistical Learning Theory; PAC-Bayes; Transfer Learning; Domain adaptation

Domaines: Machine Learning [stat.ML]; Apprentissage [cs.LG]

Pascal Germain, Amaury Habrard, François Laviolette, Emilie Morvant. A New PAC-Bayesian Perspective on Domain Adaptation. 33rd International Conference on Machine Learning (ICML 2016), Jun 2016, New York, NY, United States. ⟨hal-01307045⟩