Interpreting Robustness Proofs of Deep Neural Networks
Authors: Debangshu Banerjee, Avaljot Singh, Gagandeep Singh
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Leveraging the proposed method, we show that the robustness proofs of standard DNNs rely more on spurious input features as compared to the proofs of DNNs trained to be robust. Robustness proofs of the provably robust DNNs filter out a larger number of spurious input features as compared to adversarially trained DNNs, sometimes even leading to the pruning of semantically meaningful input features. The proofs for the DNNs combining adversarial and provably robust training tend to achieve the middle ground. ... For evaluation we use convolutional networks trained on two popular datasets MNIST (Le Cun et al., 1989) CIFAR-10 (Krizhevsky, 2009) shown in Table 1. |
| Researcher Affiliation | Collaboration | Debangshu Banerjee1, Avaljot Singh1, Gagandeep Singh1,2 1University of Illinois Urbana-Champaign, 2VMware Research |
| Pseudocode | Yes | Algorithm 1 Approximate minimum proof feature extraction |
| Open Source Code | Yes | Pro FIt code is available at https://github.com/uiuc-focal-lab/Profit. |
| Open Datasets | Yes | For evaluation we use convolutional networks trained on two popular datasets MNIST (Le Cun et al., 1989) CIFAR-10 (Krizhevsky, 2009) shown in Table 1. |
| Dataset Splits | No | The paper uses the test set for evaluation but does not explicitly describe a separate validation split or how data was partitioned into training, validation, and test sets. It mentions using the first 500 images from test sets for defining input specification ϕ, which implies a test set usage, but no specific split ratios or methods for all three sets are provided. |
| Hardware Specification | Yes | We run all experiments on a 16-core 12th-gen i7 machine with 16 GB of RAM. |
| Software Dependencies | No | The paper mentions using "the state-of-the-art incomplete verifier α-Crown (Xu et al., 2021)" and other verifiers like "Deep Z (Singh et al., 2018a), Crown (Zhang et al., 2018) and state-of-the-art complete verifier α, β-Crown (Wang et al., 2021b)" and that α-CROWN is from "auto-Li RPA (Xu et al., 2020) toolbox". However, it does not specify software version numbers for any of these tools or other dependencies. |
| Experiment Setup | Yes | ϵtrain = 0.1 and ϵtrain = 8/255 are used while training all robustly trained MNIST and CIFAR-10 networks respectively. Unless specified otherwise, the proofs are generated by running the state-of-the-art incomplete verifier α-Crown (Xu et al., 2021). |