Polynomially Over-Parameterized Convolutional Neural Networks Contain Structured Strong Winning Lottery Tickets
Authors: Arthur da Cunha, Francesco D'Amore, Natale
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | In this paper, we overcome these limitations: we leverage recent advances in the multidimensional generalisation of the Random Subset-Sum Problem and obtain a variant that admits the stochastic dependencies that arise when addressing structured pruning in the SLTH. We apply this result to prove, for a wide class of random Convolutional Neural Networks, the existence of structured subnetworks that can approximate any sufficiently smaller network. This result provides the first sub-exponential bound around the SLTH for structured pruning, opening up new avenues for further research on the hypothesis and contributing to the understanding of the role of over-parameterization in deep learning. |
| Researcher Affiliation | Academia | Arthur da Cunha Université Côte d Azur, Inria, CNRS, I3S Aarhus University Aarhus, Denmark dac@cs.au.dk Francesco d Amore Aalto University Bocconi University Espoo, Finland francesco.damore@aalto.fi Emanuele Natale Université Côte d Azur, Inria, CNRS, I3S Sophia Antipolis, France emanuele.natale@inria.fr |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. It focuses on mathematical proofs and theorems. |
| Open Source Code | No | The paper does not provide concrete access to source code for the methodology described. There are no statements about code release or links to repositories. |
| Open Datasets | No | This is a theoretical paper and does not involve experimental evaluation on datasets, thus it does not mention publicly available datasets for training. |
| Dataset Splits | No | This is a theoretical paper and does not involve experimental evaluation on datasets, thus it does not provide dataset split information for training, validation, or testing. |
| Hardware Specification | No | This is a theoretical paper focused on mathematical proofs; it does not describe any hardware used for running experiments. |
| Software Dependencies | No | This is a theoretical paper focused on mathematical proofs; it does not mention specific software dependencies with version numbers for experimental setup. |
| Experiment Setup | No | This is a theoretical paper and does not include empirical experiments, thus it does not provide details about experimental setup, hyperparameters, or training configurations. |