A Stochastic Multi-layer Algorithm for Semi-Discrete Optimal Transport with Applications to Texture Synthesis and Style Transfer

Arthur Leclaire (1) , Julien Rabin (2)

1 IMB - Institut de Mathématiques de Bordeaux
2 Equipe Image - Laboratoire GREYC - UMR6072

This paper investigates a new stochastic algorithm to approximate semi-discrete optimal transport for large-scale problem, i.e. in high dimension and for a large number of points. The proposed technique relies on a hierarchical decomposition of the target discrete distribution and the transport map itself. A stochastic optimization algorithm is derived to estimate the parameters of the corresponding multi-layer weighted nearest neighbor model. This model allows for fast evaluation during synthesis and training, for which it exhibits faster empirical convergence. Several applications to patch-based image processing are investigated: texture synthesis, texture inpainting, and style transfer. The proposed models compare favorably to the state of the art, either in terms of image quality, computation time, or regarding the number of parameters. Additionally, they do not require any pixel-based optimization or training on a large dataset of natural images.

Domaines: Traitement du signal et de l'image [eess.SP]; Optimisation et contrôle [math.OC]; Statistiques [math.ST]

Arthur Leclaire, Julien Rabin. A Stochastic Multi-layer Algorithm for Semi-Discrete Optimal Transport with Applications to Texture Synthesis and Style Transfer. Journal of Mathematical Imaging and Vision, 2020, 63, pp.282-308. ⟨10.1007/s10851-020-00975-4⟩. ⟨hal-02331068v2⟩