Skip to Main content Skip to Navigation
Conference papers

PSM-nets: Compressing Neural Networks with Product of Sparse Matrices

Abstract : Over-parameterization of neural networks is a well known issue that comes along with their great performance. Among the many approaches proposed to tackle this problem, low-rank tensor decompositions are largely investigated to compress deep neural networks. Such techniques rely on a low-rank assumption of the layer weight tensors that does not always hold in practice. Following this observation, this paper studies sparsity inducing techniques to build new sparse matrix product layers for high-rate neural networks compression. Specifically, we explore recent advances in sparse optimization to replace each layer's weight matrix, either convolutional or fully connected, by a product of sparse matrices. Our experiments validate that our approach provides a better compression-accuracy trade-off than most popular low-rank-based compression techniques.
Complete list of metadata
Contributor : Luc Giffon <>
Submitted on : Wednesday, February 24, 2021 - 7:47:15 PM
Last modification on : Thursday, July 1, 2021 - 3:36:05 AM
Long-term archiving on: : Tuesday, May 25, 2021 - 7:03:03 PM


Files produced by the author(s)


  • HAL Id : hal-03151539, version 1



Luc Giffon, Stéphane Ayache, Hachem Kadri, Thierry Artières, Ronan Sicre. PSM-nets: Compressing Neural Networks with Product of Sparse Matrices. IJCNN, Jul 2021, Virtual Event, United States. ⟨hal-03151539⟩



Record views


Files downloads