Deep Sequential Neural Network

Ludovic Denoyer 1 Patrick Gallinari 1
1 MLIA - Machine Learning and Information Access
LIP6 - Laboratoire d'Informatique de Paris 6
Abstract : Neural Networks sequentially build high-level features through their successive layers. We propose here a new neural network model where each layer is associated with a set of candidate mappings. When an input is processed, at each layer, one mapping among these candidates is selected according to a sequential decision process. The resulting model is structured according to a DAG like architecture, so that a path from the root to a leaf node defines a sequence of transformations. Instead of considering global transformations, like in classical multilayer networks, this model allows us for learning a set of local transformations. It is thus able to process data with different characteristics through specific sequences of such local transformations, increasing the expression power of this model w.r.t a classical multilayered network. The learning algorithm is inspired from policy gradient techniques coming from the reinforcement learning domain and is used here instead of the classical back-propagation based gradient descent techniques. Experiments on different datasets show the relevance of this approach.
Type de document :
Poster
Deep Learning and Representation Learning Workshop, NIPS 2014, Dec 2014, Montreal, Canada
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01222608
Contributeur : Lip6 Publications <>
Soumis le : vendredi 30 octobre 2015 - 11:38:58
Dernière modification le : jeudi 22 novembre 2018 - 14:39:29

Lien texte intégral

Identifiants

  • HAL Id : hal-01222608, version 1
  • ARXIV : 1410.0510

Collections

Citation

Ludovic Denoyer, Patrick Gallinari. Deep Sequential Neural Network. Deep Learning and Representation Learning Workshop, NIPS 2014, Dec 2014, Montreal, Canada. 〈hal-01222608〉

Partager

Métriques

Consultations de la notice

186