A PAC-Bayesian Approach for Domain Adaptation with Specialization to Linear Classifiers

Pascal Germain (1) , Amaury Habrard (2) , François Laviolette (3) , Emilie Morvant (2, 4, 5)

1 GRAAL
2 LHC - Laboratoire Hubert Curien
3 Département d'informatique et de génie logiciel [Québec]
4 QARMA - éQuipe AppRentissage et MultimediA [Marseille]
5 LIF - Laboratoire d'informatique Fondamentale de Marseille

We provide a first PAC-Bayesian analysis for domain adaptation (DA) which arises when the learning and test distributions differ. It relies on a novel distribution pseudodistance based on a disagreement averaging. Using this measure, we derive a PAC-Bayesian DA bound for the stochastic Gibbs classifier. This bound has the advantage of being directly optimizable for any hypothesis space. We specialize it to linear classifiers, and design a learning algorithm which shows interesting results on a synthetic problem and on a popular sentiment annotation task. This opens the door to tackling DA tasks by making use of all the PAC-Bayesian tools.

Mots clés: PAC-Bayesian Learning; Domain Adaptation

Domaines: Machine Learning [stat.ML]

Pascal Germain, Amaury Habrard, François Laviolette, Emilie Morvant. A PAC-Bayesian Approach for Domain Adaptation with Specialization to Linear Classifiers. International Conference on Machine Learning 2013, Jun 2013, Atlanta, United States. pp.738-746. ⟨hal-00822685⟩