A Quantitative Geometric Approach to Neural-Network Smoothness
Authors: Zi Wang, Gautam Prakriya, Somesh Jha
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our empirical evaluation demonstrates that they are more scalable and precise than existing tools on Lipschitz constant estimation for ℓ -perturbations. We implement the algorithms and name the tool Geo LIP. We empirically validate our theoretical claims, and compare Geo LIP with existing methods to estimate FGL for ℓ -perturbations. The result shows that Geo LIP provides a tighter bound (20%-60% improvements) than existing tools on small networks, and much better results than the naive matrix-norm-product method for deep networks, which existing tools cannot handle (Section 6). |
| Researcher Affiliation | Academia | Zi Wang University of Wisconsin-Madison zw@cs.wisc.edu Gautam Prakriya The Chinese University of Hong Kong gautamprakriya@gmail.com Somesh Jha University of Wisconsin-Madison jha@cs.wisc.edu |
| Pseudocode | No | The paper describes algorithms and derivations, but it does not include any structured pseudocode blocks or clearly labeled algorithm figures. |
| Open Source Code | Yes | Our code is available at https://github.com/z1w/Geo LIP. Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [Yes] We included the code, data, and instructions as a URL. |
| Open Datasets | Yes | We will run Geo LIP and existing tools that measure the ℓ -FGL on various feedforward neural networks trained with the MNIST dataset (Le Cun and Cortes, 2010). In terms of dataset, we only used the standard MNIST, and cited the creators. |
| Dataset Splits | Yes | We run the experiments on fully-connected feed-forward neural networks, trained with the MNIST dataset for 10 epochs using the ADAM optimizer (Kingma and Ba, 2015). All the trained networks have accuracy greater than 92% on the test data. Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] We included major experimental setting in Section 6.1, and detailed specification in the appendix. |
| Hardware Specification | No | The paper states 'Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] This was provided in the appendix.' However, the provided text does not include the appendix, so no specific hardware details are present in the extract. |
| Software Dependencies | Yes | We have implemented the algorithms using MATLAB (MATLAB, 2021), the CVX toolbox (CVX Research, 2020) and MOSEK solver (Ap S, 2019), and name the tool Geo LIP. Li Popt is based on the Python Gurobi solver (Gurobi Optimization, LLC, 2022). |
| Experiment Setup | Yes | We run the experiments on fully-connected feed-forward neural networks, trained with the MNIST dataset for 10 epochs using the ADAM optimizer (Kingma and Ba, 2015). For two-layer networks, we use 8, 16, 64, 128, 256 hidden nodes. For multiple-layer networks, we consider 3, 7, 8-layer networks, and each hidden layer has 64 Re LU units. Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] We included major experimental setting in Section 6.1, and detailed specification in the appendix. |