Residual Networks as Flows of Diffeomorphisms - Archive ouverte HAL Accéder directement au contenu
Article Dans Une Revue Journal of Mathematical Imaging and Vision Année : 2019

Residual Networks as Flows of Diffeomorphisms

Résumé

This paper addresses the understanding and characterization of residual networks (ResNet), which are among the state-of-the-art deep learning architectures for a variety of supervised learning problems. We focus on the mapping component of ResNets, which map the embedding space towards a new unknown space where the prediction or classification can be stated according to linear criteria. We show that this mapping component can be regarded as the numerical implementation of continuous flows of diffeomorphisms governed by ordinary differential equations. Especially, ResNets with shared weights are fully characterized as numerical approximation of exponential diffeomorphic operators. We stress both theoretically and numerically the relevance of the enforcement of diffeormorphic properties and the importance of numerical issues to make consistent the continuous formulation and the discretized ResNet implementation. We further discuss the resulting theoretical and computational insights on ResNet architectures.
Fichier principal
Vignette du fichier
JMIV_ResNet.pdf (2.82 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01796729 , version 1 (24-05-2018)
hal-01796729 , version 2 (13-09-2019)

Identifiants

Citer

François Rousseau, Lucas Drumetz, Ronan Fablet. Residual Networks as Flows of Diffeomorphisms. Journal of Mathematical Imaging and Vision, 2019, ⟨10.1007/s10851-019-00890-3⟩. ⟨hal-01796729v2⟩
296 Consultations
996 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More