Heterogeneous Wasserstein Discrepancy for Incomparable Distributions

Mokhtar Z. Alaya (1) , Gilles Gasso (2) , Maxime Berar (2) , Alain Rakotomamonjy (3, 2)

1 LMAC - Laboratoire de Mathématiques Appliquées de Compiègne
2 LITIS - Laboratoire d'Informatique, de Traitement de l'Information et des Systèmes
3 Criteo AI Lab

Optimal Transport (OT) metrics allow for defining discrepancies between two probability measures. Wasserstein distance is for longer the celebrated OT-distance frequently used in the literature, which seeks probability distributions to be supported on the same metric space. Because of its high computational complexity, several approximate Wasserstein distances have been proposed based on entropy regularization or on slicing, and one-dimensional Wassserstein computation. In this paper, we propose a novel extension of Wasserstein distance to compare two incomparable distributions, that hinges on the idea of distributional slicing, embeddings, and on computing the closed-form Wassertein distance between the sliced distributions. We provide a theoretical analysis of this new divergence, called heterogeneous Wasserstein discrepancy (HWD), and we show that it preserves several interesting properties including rotation-invariance. We show that the embeddings involved in HWD can be efficiently learned. Finally, we provide a large set of experiments illustrating the behavior of HWD as a divergence in the context of generative modeling and in query framework.

Domaines: Machine Learning [stat.ML]; Théorie [stat.TH]

Mokhtar Z. Alaya, Gilles Gasso, Maxime Berar, Alain Rakotomamonjy. Heterogeneous Wasserstein Discrepancy for Incomparable Distributions. 2021. ⟨hal-03250633v2⟩