Relational DNN Verification With Cross Executional Bound Refinement
Authors: Debangshu Banerjee, Gagandeep Singh
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We perform extensive experiments on popular datasets, multiple DNNs (standard and robustly trained), and multiple relational properties showcasing that RACoon significantly outperforms the current SOTA baseline.1 |
| Researcher Affiliation | Collaboration | 1Department of Computer Science, University of Illinois Urbana-Champaign, USA 2VMware Research, USA. |
| Pseudocode | Yes | Algorithm 1 RACoon |
| Open Source Code | Yes | Code at https://github.com/uiuc-focal-lab/RACoon |
| Open Datasets | Yes | We perform extensive experiments on popular datasets, multiple DNNs (standard and robustly trained), and multiple relational properties. We used Conv Small PGD and Diff AI DNNs trained on MNIST and CIFAR10 for this experiment. |
| Dataset Splits | No | The paper mentions 'k randomly selected inputs' for experiments and refers to 'training distribution' for UAP, but does not provide details on specific training/validation/test splits, percentages, or absolute sample counts for data partitioning to reproduce the experiment. |
| Hardware Specification | Yes | We use a single NVIDIA A100-PCI GPU with 40 GB RAM for bound refinement and an Intel(R) Xeon(R) Silver 4214R CPU @ 2.40GHz with 64 GB RAM for MILP optimization. |
| Software Dependencies | Yes | We implemented our method in Python with Pytorch V1.11 and used Gurobi V10.0.3 as an off-the-shelf MILP solver. |
| Experiment Setup | Yes | We run 20 iterations of Adam on each set of executions. For each relational property, we use k0 = 6 and k1 = 4 for deciding which set of executions to consider for cross-execution refinement as discussed in section 5. |