Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Model Selection for Bayesian Autoencoders

Abstract : We develop a novel method for carrying out model selection for Bayesian autoencoders (BAEs) by means of prior hyper-parameter optimization. Inspired by the common practice of type-II maximum likelihood optimization and its equivalence to Kullback-Leibler divergence minimization, we propose to optimize the distributional sliced-Wasserstein distance (DSWD) between the output of the autoencoder and the empirical data distribution. The advantages of this formulation are that we can estimate the DSWD based on samples and handle high-dimensional problems. We carry out posterior estimation of the BAE parameters via stochastic gradient Hamiltonian Monte Carlo and turn our BAE into a generative model by fitting a flexible Dirichlet mixture model in the latent space. Consequently, we obtain a powerful alternative to variational autoencoders, which are the preferred choice in modern applications of autoencoders for representation learning with uncertainty. We evaluate our approach qualitatively and quantitatively using a vast experimental campaign on a number of unsupervised learning tasks and show that, in small-data regimes where priors matter, our approach provides state-of-the-art results, outperforming multiple competitive baselines.
Document type :
Preprints, Working Papers, ...
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-03344782
Contributor : Centre de Documentation Eurecom Connect in order to contact the contributor
Submitted on : Wednesday, September 15, 2021 - 11:21:19 AM
Last modification on : Thursday, September 16, 2021 - 3:40:44 AM

Links full text

Identifiers

  • HAL Id : hal-03344782, version 1
  • ARXIV : 2106.06245

Collections

Citation

Ba-Hien Tran, Simone Rossi, Dimitrios Milios, Pietro Michiardi, Edwin V. Bonilla, et al.. Model Selection for Bayesian Autoencoders. 2021. ⟨hal-03344782⟩

Share

Metrics

Record views

16