Skip to Main content Skip to Navigation
Conference papers

Proving the Strong Lottery Ticket Hypothesis for Convolutional Neural Networks

Arthur da Cunha 1 Emanuele Natale 1 Laurent Viennot 2 
1 COATI - Combinatorics, Optimization and Algorithms for Telecommunications
CRISAM - Inria Sophia Antipolis - Méditerranée , Laboratoire I3S - COMRED - COMmunications, Réseaux, systèmes Embarqués et Distribués
Abstract : The lottery ticket hypothesis states that a randomly-initialized neural network contains a small subnetwork which, when trained in isolation, can compete with the performance of the original network. Recent theoretical works proved an even stronger version: every sufficiently overparameterized (dense) neural network contains a subnetwork that, even without training, achieves accuracy comparable to that of the trained large network. These works left as an open problem to extend the result to convolutional neural networks (CNNs). In this work we provide such generalization by showing that, with high probability, it is possible to approximate any CNN by pruning a random CNN whose size is larger by a logarithmic factor.
Document type :
Conference papers
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-03548226
Contributor : Arthur da Cunha Connect in order to contact the contributor
Submitted on : Saturday, January 29, 2022 - 11:42:02 PM
Last modification on : Thursday, June 9, 2022 - 3:17:18 AM
Long-term archiving on: : Saturday, April 30, 2022 - 6:13:41 PM

File

SLTHforCNNs.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03548226, version 1

Collections

Citation

Arthur da Cunha, Emanuele Natale, Laurent Viennot. Proving the Strong Lottery Ticket Hypothesis for Convolutional Neural Networks. ICLR 2022 - 10th International Conference on Learning Representations, Apr 2022, Virtual, France. ⟨hal-03548226⟩

Share

Metrics

Record views

101

Files downloads

147