Unbiased online recurrent optimization

Corentin Tallec 1, 2 Yann Ollivier 1
2 TAU - TAckling the Underspecified
LRI - Laboratoire de Recherche en Informatique, UP11 - Université Paris-Sud - Paris 11, Inria Saclay - Ile de France, CNRS - Centre National de la Recherche Scientifique : UMR8623
Abstract : The novel Unbiased Online Recurrent Optimization (UORO) algorithm allows for online learning of general recurrent computational graphs such as recurrent network models. It works in a streaming fashion and avoids backtracking through past activations and inputs. UORO is computationally as costly as Truncated Backpropagation Through Time (truncated BPTT), a widespread algorithm for online learning of recurrent networks Jaeger (2002). UORO is a modification of NoBackTrack Ollivier et al. (2015) that bypasses the need for model sparsity and makes implementation easy in current deep learning frameworks, even for complex models. Like NoBackTrack, UORO provides unbiased gradient estimates; unbiasedness is the core hypothesis in stochastic gradient descent theory, without which convergence to a local optimum is not guaranteed. On the contrary, truncated BPTT does not provide this property, leading to possible divergence. On synthetic tasks where truncated BPTT is shown to diverge, UORO converges. For instance, when a parameter has a positive short-term but negative long-term influence , truncated BPTT diverges unless the truncation span is very significantly longer than the intrinsic temporal range of the interactions, while UORO performs well thanks to the unbiasedness of its gradients.
Type de document :
Communication dans un congrès
International Conference On Learning Representation, Apr 2018, Vancouver, Canada
Liste complète des métadonnées

Contributeur : Corentin Tallec <>
Soumis le : lundi 7 janvier 2019 - 17:43:09
Dernière modification le : jeudi 7 février 2019 - 15:40:09


Fichiers produits par l'(les) auteur(s)


  • HAL Id : hal-01972587, version 1


Corentin Tallec, Yann Ollivier. Unbiased online recurrent optimization. International Conference On Learning Representation, Apr 2018, Vancouver, Canada. 〈hal-01972587〉



Consultations de la notice


Téléchargements de fichiers