Tensorizing Neural Networks

Abstract : Deep neural networks currently demonstrate state-of-the-art performance in several domains. At the same time, models of this class are very demanding in terms of computational resources. In particular, a large amount of memory is required by commonly used fully-connected layers, making it hard to use the models on low-end devices and stopping the further increase of the model size. In this paper we convert the dense weight matrices of the fully-connected layers to the Tensor Train format such that the number of parameters is reduced by a huge factor and at the same time the expressive power of the layer is preserved. In particular, for the Very Deep VGG networks we report the compression factor of the dense weight matrix of a fully-connected layer up to 200000 times leading to the compression factor of the whole network up to 7 times.
Type de document :
Communication dans un congrès
The 29-th Conference on Natural Information Processing Systems (NIPS), Dec 2015, Montréal, Canada. Advances in Neural Information Processing Systems
Liste complète des métadonnées

https://hal.archives-ouvertes.fr/hal-01237600
Contributeur : Anton Osokin <>
Soumis le : jeudi 3 décembre 2015 - 14:57:36
Dernière modification le : vendredi 25 mai 2018 - 12:02:06

Lien texte intégral

Identifiants

  • HAL Id : hal-01237600, version 1
  • ARXIV : 1509.06569

Collections

Citation

Alexander Novikov, Dmitry Podoprikhin, Anton Osokin, Dmitry Vetrov. Tensorizing Neural Networks. The 29-th Conference on Natural Information Processing Systems (NIPS), Dec 2015, Montréal, Canada. Advances in Neural Information Processing Systems. 〈hal-01237600〉

Partager

Métriques

Consultations de la notice

190