Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Backprop Diffusion is Biologically Plausible

Marco Gori 1, 2 Alessandro Betti 2
1 MAASAI - Modèles et algorithmes pour l’intelligence artificielle
CRISAM - Inria Sophia Antipolis - Méditerranée , Laboratoire I3S - SPARKS - Scalable and Pervasive softwARe and Knowledge Systems, UNS - Université Nice Sophia Antipolis (... - 2019), JAD - Laboratoire Jean Alexandre Dieudonné
Abstract : The Backpropagation algorithm relies on the abstraction of using a neural model that gets rid of the notion of time, since the input is mapped instantaneously to the output. In this paper, we claim that this abstraction of ignoring time, along with the abrupt input changes that occur when feeding the training set, are in fact the reasons why, in some papers, Backprop biological plausibility is regarded as an arguable issue. We show that as soon as a deep feedforward network operates with neurons with time-delayed response, the backprop weight update turns out to be the basic equation of a biologically plausible diffusion process based on forward-backward waves. We also show that such a process very well approximates the gradient for inputs that are not too fast with respect to the depth of the network. These remarks somewhat disclose the diffusion process behind the backprop equation and leads us to interpret the corresponding algorithm as a degeneration of a more general diffusion process that takes place also in neural networks with cyclic connections.
Document type :
Preprints, Working Papers, ...
Complete list of metadatas

Cited literature [12 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-02878574
Contributor : Marco Gori <>
Submitted on : Tuesday, June 23, 2020 - 6:28:43 PM
Last modification on : Friday, June 26, 2020 - 10:18:42 AM

File

1912.04635.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-02878574, version 1

Collections

Citation

Marco Gori, Alessandro Betti. Backprop Diffusion is Biologically Plausible. 2020. ⟨hal-02878574⟩

Share

Metrics

Record views

39

Files downloads

19