Sign Bits Are All You Need for Black-Box Attacks
Authors: Abdullah Al-Dujaili, Una-May O'Reilly
ICLR 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our approach on a rigorous set of experiments on both, standard and adversarially hardened models. All other previous works on this topic have published their results on a subset of the datasets and threat models we experimentally validate in this work. |
| Researcher Affiliation | Academia | Abdullah Al-Dujaili CSAIL, MIT Cambridge, MA 02139 aldujail@mit.edu Una-May O Reilly CSAIL, MIT Cambridge, MA 02139 unamay@csail.mit.edu |
| Pseudocode | Yes | Algorithm 1 Sign Hunter |
| Open Source Code | Yes | Code and data for the experiments can be found at https://bit.ly/3ac IHo Q. |
| Open Datasets | Yes | We evaluate Sign Hunter and compare it with established algorithms from the literature... on the MNIST, CIFAR10, and IMAGENET datasets. ...For MNIST and CIFAR10, the naturally trained models from (Madry et al., 2017) s MNIST and CIFAR10 challenges are used. For IMAGENET, Tensor Flow s Inception (v3) model is used. |
| Dataset Splits | Yes | Each attacker is given a budget of 10, 000 oracle queries per attack attempt and is evaluated on 1000 images from the test sets of MNIST, CIFAR10, and the validation set of IMAGENET. |
| Hardware Specification | Yes | All experiments were run on a CUDA-enabled NVIDIA Tesla V100 16GB. |
| Software Dependencies | No | The paper mentions 'Tensor Flow s Inception (v3) model is used' but does not specify its version or any other software dependencies with version numbers, such as Python, PyTorch, or CUDA versions. |
| Experiment Setup | Yes | Experiments Setup. Our experiment setup is similar to (Ilyas et al., 2019). ...Hyperparameters Setup. While Sign Hunter does not have any hyperparameters, to fairly compare it with the other algorithms, we tuned their hyperparameters starting with the default values reported by the corresponding authors. ...Details on the hyperparameter setup are available in Appendix C. (Appendix C includes Tables 4-7 with specific hyperparameter values for NES, ZO-Sign SGD, Bandits T D, and Sign Hunter). |