Overcoming the Convex Barrier for Simplex Inputs
Authors: Harkirat Singh Behl, M. Pawan Kumar, Philip Torr, Krishnamurthy Dvijotham
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We establish the scalability of our overall approach via the specification of ℓ1 robustness for CIFAR-10 and MNIST classification, where our approach improves the state of the art verified accuracy by up to 14.4%. |
| Researcher Affiliation | Collaboration | Harkirat Singh University of Oxford harkirat@robots.ox.ac.uk M. Pawan Kumar Deep Mind mpawan@deepmind.com Philip H.S. Torr University of Oxford phst@robots.ox.ac.uk Krishnamurthy (Dj) Dvijotham Deep Mind dvij@google.com |
| Pseudocode | Yes | Algorithm 1 Simplex Verify |
| Open Source Code | No | The paper states 'We used the publicly available training implementation of [Ding et al., 2019]...The code is made available under the LGPL License online 1. 1https://github.com/Borealis AI/advertorch.' This refers to a third-party library used for training, not the authors' own method. |
| Open Datasets | Yes | We evaluate the effectiveness of various methods for incomplete verification on the MNIST [Lecun] and CIFAR-10 [Krizhevsky and Hinton, 2009] datasets. ... The MNIST and CIFAR-10 datasets are widely used in the machine learning community, and are available under the creator s consent and MIT license respectively. ... For this experiment we verify the robustness of models on the UPMC FOOD-101 dataset [Wang et al., 2015] ... It is made available by the creators consent online 2. 2http://visiir.lip6.fr/ |
| Dataset Splits | No | The paper mentions evaluating on test sets but does not specify training/validation/test dataset splits, exact percentages, or sample counts for reproduction. |
| Hardware Specification | Yes | Both the methods are run on 4 CPU threads on an Intel(R) Core(TM) i7-4960X CPU @ 3.60GHz processor. ... Both the Li RPA based solvers use Adam [Kingma and Ba, 2015] for updating the weighting vectors a, and are run on a single Nvidia Titan Xp GPU. |
| Software Dependencies | No | The paper mentions Gurobi and advertorch but does not provide specific version numbers for these or any other software dependencies. |
| Experiment Setup | Yes | The models are trained using the SLIDE attack (sparse ℓ1-descent attack) from Tramer and Boneh [2019] with ϵ = 0.3 for all networks except the VNN-comp big network, which is trained with ϵ = 0.05. ... We verify robustness against input perturbations lying in ℓ1 norm ball with ϵ = 0.35 for the MNIST network, ϵ = 0.2 for the VNN-comp big network and ϵ = 0.5 for all the other CIFAR-10 networks. ... All the methods use the same intermediate bounds, which are computed using Opt-Lirpa Planet run for 20 iterations. |