Certifying Geometric Robustness of Neural Networks
Authors: Mislav Balunovic, Maximilian Baader, Gagandeep Singh, Timon Gehr, Martin Vechev
NeurIPS 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We implemented our certification method in a system called DEEPG. First, we demonstrate that DEEPG can certify robustness to significantly more complex transformations than both prior work and traditional bound propagation approaches based on relational abstractions. Second, we experimentally show that our method requires relatively small number of samples to converge to the optimal linear constraints. Third, we investigate the effectiveness of a variety of training methods to train a network provably robust to geometric transformations. Finally, we demonstrate that DEEPG is scalable and can certify geometric robustness for large architectures. |
| Researcher Affiliation | Academia | Mislav Balunovi c, Maximilian Baader, Gagandeep Singh, Timon Gehr, Martin Vechev Department of Computer Science ETH Zurich {mislav.balunovic, mbaader, gsingh, timon.gehr, martin.vechev}@inf.ethz.ch |
| Pseudocode | Yes | Algorithm 1 Lipschitz Optimization with Bound Refinement |
| Open Source Code | Yes | We make DEEPG publicly available at https://github.com/eth-sri/deepg/. |
| Open Datasets | Yes | We evaluate on image recognition datasets: MNIST [36], Fashion-MNIST [39] and CIFAR-10 [40]. |
| Dataset Splits | No | We evaluate on image recognition datasets: MNIST [36], Fashion-MNIST [39] and CIFAR-10 [40]. For each dataset, we randomly select 100 images from the test set to certify. |
| Hardware Specification | Yes | All experiments except the one with large networks were performed on a desktop PC with 2 Ge Force RTX 2080 Ti GPU-s and 16-core Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz. |
| Software Dependencies | No | The paper discusses the use of systems like DEEPG and Deep Poly, but it does not specify versions for any software dependencies (e.g., programming languages, libraries, or frameworks). |
| Experiment Setup | Yes | Details of these architectures are provided in Appendix B.2. We certify robustness to composition of transformations such as rotation, translation, scaling, shearing and changes in brightness and contrast. These transformations are formally defined in Appendix A.1. ... We provide runtime analysis of the experiments and all hyperparameters used for certification in Appendix B.2. |