PAC-Bayes and Domain Adaptation

Pascal Germain 1 Amaury Habrard 2 François Laviolette 3 Emilie Morvant 2
1 MODAL - MOdel for Data Analysis and Learning
Inria Lille - Nord Europe, LPP - Laboratoire Paul Painlevé - UMR 8524, CERIM - Santé publique : épidémiologie et qualité des soins-EA 2694, Polytech Lille - École polytechnique universitaire de Lille, Université de Lille, Sciences et Technologies
Abstract : We provide two main contributions in PAC-Bayesian theory for domain adaptation where the objective is to learn, from a source distribution, a well-performing majority vote on a different, but related, target distribution. Firstly, we propose an improvement of the previous approach we proposed in Germain et al. (2013), which relies on a novel distribution pseudodistance based on a disagreement averaging, allowing us to derive a new tighter domain adaptation bound for the target risk. While this bound stands in the spirit of common domain adaptation works, we derive a second bound (recently introduced in Germain et al., 2016) that brings a new perspective on domain adaptation by deriving an upper bound on the target risk where the distributions' divergence—expressed as a ratio—controls the trade-off between a source error measure and the target voters' disagreement. We discuss and compare both results, from which we obtain PAC-Bayesian generalization bounds. Furthermore, from the PAC-Bayesian specialization to linear classifiers, we infer two learning algorithms, and we evaluate them on real data.
Document type :
Preprints, Working Papers, ...
Complete list of metadatas

https://hal.archives-ouvertes.fr/hal-01563152
Contributor : Emilie Morvant <>
Submitted on : Tuesday, November 6, 2018 - 1:34:27 PM
Last modification on : Friday, April 19, 2019 - 4:54:59 PM
Long-term archiving on : Thursday, February 7, 2019 - 1:05:50 PM

Files

main.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01563152, version 2
  • ARXIV : 1707.05712

Citation

Pascal Germain, Amaury Habrard, François Laviolette, Emilie Morvant. PAC-Bayes and Domain Adaptation. 2018. ⟨hal-01563152v2⟩

Share

Metrics

Record views

120

Files downloads

164