HAL will be down for maintenance from Friday, June 10 at 4pm through Monday, June 13 at 9am. More information
Skip to Main content Skip to Navigation
Conference papers

Weight Reparametrization for Budget-Aware Network Pruning

Abstract : Pruning seeks to design lightweight architectures by removing redundant weights in overparameterized networks. Most of the existing techniques first remove structured subnetworks (filters, channels,...) and then fine-tune the resulting networks to maintain a high accuracy. However, removing a whole structure is a strong topological prior and recovering the accuracy, with fine-tuning, is highly cumbersome. In this paper, we introduce an "end-to-end" lightweight network design that achieves training and pruning simultaneously without fine-tuning. The design principle of our method relies on reparametrization that learns not only the weights but also the topological structure of the lightweight sub-network. This reparametrization acts as a prior (or regularizer) that defines pruning masks implicitly from the weights of the underlying network, without increasing the number of training parameters. Sparsity is induced with a budget loss that provides an accurate pruning. Extensive experiments conducted on the CIFAR10 and the TinyImageNet datasets, using standard architectures (namely Conv4, VGG19 and ResNet18), show compelling results without fine-tuning.
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-03431309
Contributor : Hichem Sahbi Connect in order to contact the contributor
Submitted on : Tuesday, November 16, 2021 - 3:55:13 PM
Last modification on : Wednesday, January 19, 2022 - 2:08:02 PM
Long-term archiving on: : Thursday, February 17, 2022 - 8:06:12 PM

File

paperA.pdf
Files produced by the author(s)

Identifiers

Citation

Robin Dupont, Hichem Sahbi, Guillaume Michel. Weight Reparametrization for Budget-Aware Network Pruning. IEEE International Conference on Image Processing (ICIP), Sep 2021, Anchorage, AK (virtual), United States. pp.789-793, ⟨10.1109/ICIP42928.2021.9506265⟩. ⟨hal-03431309⟩

Share

Metrics

Record views

22

Files downloads

23