Lipschitz constant estimation of Neural Networks via sparse polynomial optimization
Authors: Fabian Latorre, Paul Rolland, Volkan Cevher
ICLR 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct experiments on networks with random weights as well as networks trained on MNIST, showing that in the particular case of the ℓ -Lipschitz constant, our approach yields superior estimates, compared to baselines available in the literature. |
| Researcher Affiliation | Academia | Fabian Latorre, Paul Rolland and Volkan Cevher EPFL, Switzerland firstname.lastname@epfl.ch |
| Pseudocode | Yes | Algorithm 1 Li Popt for ELU activations and sparsity pattern |
| Open Source Code | No | The paper states: "For training we used the code from this reference. It is publicly available in https://github.com/mightydeveloper/Deep-Compression-PyTorch". This refers to a third-party code used for a component of their work (pruning strategy), not the source code for their proposed Li Popt methodology. |
| Open Datasets | Yes | We conduct experiments on networks with random weights as well as networks trained on MNIST (Lecun et al., 1998). |
| Dataset Splits | No | The paper mentions using MNIST and describes network architectures but does not specify the train/validation/test splits used for the dataset. |
| Hardware Specification | Yes | All methods run on a single machine with Core i7 2.8Ghz quad-core processor and 16Gb of RAM. |
| Software Dependencies | No | The paper states: "Li Popt uses the Gurobi LP solver, while SDP uses Mosek.". While specific software is named, no version numbers are provided for these solvers or any other software components. |
| Experiment Setup | Yes | For different configurations of width and sparsity, we generate 10 random networks and average the obtained Lipschitz bounds. The architecture we use is a fully connected network with two hidden layers with 300 and 100 neurons respectively. We were able to remove 95% of the weights. |