Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Taking Apart Autoencoders: How do They Encode Geometric Shapes ?

Abstract : We study the precise mechanisms which allow autoencoders to encode and decode a simple geometric shape, the disk. In this carefully controlled setting, we are able to describe the specific form of the optimal solution to the minimisation problem of the training step. We show that the autoencoder indeed approximates this solution during training. Secondly, we identify a clear failure in the generali-sation capacity of the autoencoder, namely its inability to interpolate data. Finally, we explore several regularisation schemes to resolve the generalisation problem. Given the great attention that has been recently given to the generative capacity of neural networks, we believe that studying in depth simple geometric cases sheds some light on the generation process and can provide a minimal requirement experimental setup for more complex architectures.
Document type :
Preprints, Working Papers, ...
Complete list of metadatas

Cited literature [19 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01676326
Contributor : Alasdair Newson <>
Submitted on : Friday, January 5, 2018 - 2:32:50 PM
Last modification on : Friday, July 31, 2020 - 10:44:11 AM

File

autoencoders_preprint.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01676326, version 1

Citation

Alasdair Newson, Andrés Almansa, Yann Gousseau, Saïd Ladjal. Taking Apart Autoencoders: How do They Encode Geometric Shapes ?. 2018. ⟨hal-01676326⟩

Share

Metrics

Record views

542

Files downloads

1918