Towards Stable and Efficient Training of Verifiably Robust Neural Networks
Authors: Huan Zhang, Hongge Chen, Chaowei Xiao, Sven Gowal, Robert Stanforth, Bo Li, Duane Boning, Cho-Jui Hsieh
ICLR 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct large scale experiments on MNIST and CIFAR datasets, and outperform all previous linear relaxation and bound propagation based certified defenses in ℓ robustness. Notably, we achieve 7.02% verified test error on MNIST at ϵ = 0.3, and 66.94% on CIFAR-10 with ϵ = 8/255. |
| Researcher Affiliation | Collaboration | Huan Zhang1 Hongge Chen2 Chaowei Xiao3 Sven Gowal4 Robert Stanforth4 Bo Li5 Duane Boning2 Cho-Jui Hsieh1 1 UCLA 2 MIT 3 University of Michigan 4 Deep Mind 5 UIUC |
| Pseudocode | No | The paper describes the algorithms in text and mathematical formulas (e.g., Section 3.2 'The Proposed Algorithm: CROWN-IBP', 'Forward Bound Propagation in CROWN-IBP', 'Backward Bound Propagation in CROWN-IBP') but does not include a dedicated pseudocode block or algorithm box. |
| Open Source Code | Yes | Our training code for IBP and CROWN-IBP, and pre-trained models are publicly available 3. PyTorch implementation and pre-trained models: https://github.com/huanzhang12/CROWN-IBP |
| Open Datasets | Yes | We conduct large scale experiments on MNIST and CIFAR datasets |
| Dataset Splits | No | The paper discusses 'training set' and 'test set' extensively (e.g., Table 2, Section D). While it might implicitly use a validation set for hyperparameter tuning, it does not explicitly define or specify the proportions or methodology for train/validation/test splits, nor does it explicitly mention a 'validation' set or split sizes. |
| Hardware Specification | Yes | For CIFAR-10, we train using a large batch size and long training schedule on TPUs (we used TPUv2 which has half memory capacity as TPUv3 used in (Gowal et al., 2018))... All experiments are measured on a single RTX 2080 Ti GPU with 11 GB RAM except for 2 DM-Large models where we use 4 RTX 2080 Ti GPUs to speed up training. |
| Software Dependencies | No | The paper mentions 'TensorFlow' and 'PyTorch' as implementation frameworks, but it does not specify any version numbers for these or any other software libraries or dependencies. (e.g., 'TensorFlow implementation and pre-trained models', 'PyTorch implementation and pre-trained models'). |
| Experiment Setup | Yes | For MNIST... train 100 epochs (60K steps) with a batch size of 100, and use a warm-up and ramp-up duration of 2K and 10K steps. Learning rate for Adam optimizer is set to 1 × 10−3 and decayed by 10X at steps 15K and 25K... For CIFAR-10... train 3200 epochs on 32 TPU cores. We use a batch size of 1024, and a learning rate of 5 × 10−4. We warm up for 320 epochs, and ramp-up ϵ for 1600 epochs. Learning rate is reduced by 10X at epoch 2600 and 3040. We use random horizontal flips and random crops as data augmentation, and normalize images according to per-channel statistics. |