Skip to Main content Skip to Navigation
Conference papers

PSM-nets: Compressing Neural Networks with Product of Sparse Matrices

Abstract : Over-parameterization of neural networks is a well known issue that comes along with their great performance. Among the many approaches proposed to tackle this problem, low-rank tensor decompositions are largely investigated to compress deep neural networks. Such techniques rely on a low-rank assumption of the layer weight tensors that does not always hold in practice. Following this observation, this paper studies sparsity inducing techniques to build new sparse matrix product layers for high-rate neural networks compression. Specifically, we explore recent advances in sparse optimization to replace each layer's weight matrix, either convolutional or fully connected, by a product of sparse matrices. Our experiments validate that our approach provides a better compression-accuracy trade-off than most popular low-rank-based compression techniques.
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-03151539
Contributor : Luc Giffon <>
Submitted on : Wednesday, February 24, 2021 - 7:47:15 PM
Last modification on : Thursday, July 1, 2021 - 3:36:05 AM
Long-term archiving on: : Tuesday, May 25, 2021 - 7:03:03 PM

File

Palmnet_IJCNN__Copy_.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03151539, version 1

Collections

Citation

Luc Giffon, Stéphane Ayache, Hachem Kadri, Thierry Artières, Ronan Sicre. PSM-nets: Compressing Neural Networks with Product of Sparse Matrices. IJCNN, Jul 2021, Virtual Event, United States. ⟨hal-03151539⟩

Share

Metrics

Record views

128

Files downloads

166