Top-Down Regularization of Deep Belief Networks

Abstract : Designing a principled and effective algorithm for learning deep architectures is a challenging problem. The current approach involves two training phases: a fully unsupervised learning followed by a strongly discriminative optimization. We suggest a deep learning strategy that bridges the gap between the two phases, resulting in a three-phase learning procedure. We propose to implement the scheme using a method to regularize deep belief networks with top-down information. The network is constructed from building blocks of restricted Boltzmann machines learned by combining bottom-up and top-down sampled signals. A global optimization procedure that merges samples from a forward bottom-up pass and a top-down pass is used. Experiments on the MNIST dataset show improvements over the existing algorithms for deep belief networks. Object recognition results on the Caltech-101 dataset also yield competitive results.
Type de document :
Communication dans un congrès
C.J.C. Burges; L. Bottou; M. Welling; Z. Ghahramani; K.Q. Weinberger. Advances in Neural Information Processing Systems 26, Dec 2013, Lake Tahoe, United States. pp.1878-1886
Liste complète des métadonnées

Littérature citée [28 références]  Voir  Masquer  Télécharger

https://hal.archives-ouvertes.fr/hal-00947569
Contributeur : Hanlin Goh <>
Soumis le : mercredi 19 février 2014 - 14:15:00
Dernière modification le : jeudi 22 novembre 2018 - 14:11:45
Document(s) archivé(s) le : lundi 19 mai 2014 - 10:55:37

Fichier

13_NIPS.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-00947569, version 1

Collections

Citation

Hanlin Goh, Nicolas Thome, Matthieu Cord, Joo-Hwee Lim. Top-Down Regularization of Deep Belief Networks. C.J.C. Burges; L. Bottou; M. Welling; Z. Ghahramani; K.Q. Weinberger. Advances in Neural Information Processing Systems 26, Dec 2013, Lake Tahoe, United States. pp.1878-1886. 〈hal-00947569〉

Partager

Métriques

Consultations de la notice

271

Téléchargements de fichiers

419