Theoretically Principled Trade-off between Robustness and Accuracy
Authors: Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, Michael Jordan
ICML 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our proposed algorithm performs well experimentally in real-world datasets. The methodology is the foundation of our entry to the Neur IPS 2018 Adversarial Vision Challenge in which we won the 1st place out of ~2,000 submissions, surpassing the runner-up approach by 11.41% in terms of mean 2 perturbation distance. Experimentally, we show that our proposed algorithm outperforms state-of-the-art methods under both blackbox and white-box threat models. In particular, the methodology won the final round of the Neur IPS 2018 Adversarial Vision Challenge. |
| Researcher Affiliation | Collaboration | Hongyang Zhang 1 2 Yaodong Yu 3 Jiantao Jiao 4 Eric P. Xing 1 5 Laurent El Ghaoui 4 Michael I. Jordan 4 1Carnegie Mellon University 2Toyota Technological Institute at Chicago 3University of Virginia 4University of California, Berkeley 5Petuum Inc. |
| Pseudocode | Yes | Algorithm 1 Adversarial training by TRADES |
| Open Source Code | Yes | We release our code and trained models at https://github.com/ yaodongyu/TRADES. |
| Open Datasets | Yes | We verify the tightness of the established upper bound in Theorem 3.1 for binary classification problem on MNIST dataset. ... We approximate the above expectation terms by test dataset. ... MNIST setup. ... CIFAR10 setup. ... The dataset in this competition is Tiny Image Net, which consists of 550,000 data (with our data augmentation) and 200 classes. |
| Dataset Splits | No | The paper mentions training and test datasets, but it does not specify explicit validation splits or proportions (e.g., 80/10/10 split, or specific sample counts for validation). |
| Hardware Specification | No | The paper does not explicitly describe the specific hardware used, such as GPU models, CPU types, or memory specifications. It only generally refers to 'computational resources'. |
| Software Dependencies | No | The paper does not provide specific version numbers for software dependencies (e.g., Python 3.x, PyTorch 1.x). While it mentions using ResNet-18, it doesn't list the software environment versions. |
| Experiment Setup | Yes | We take perturbation = 0.1, number of iterations K = 20 and run 30 epochs on the training dataset. ... MNIST setup. We set perturbation = 0.1, perturbation step size 1 = 0.01, number of iterations K = 20, learning rate 2 = 0.01, batch size m = 128, and run 50 epochs on the training dataset. ... CIFAR10 setup. We apply Res Net-18 (He et al., 2016) for classification. The output size of the last layer is 10. We set perturbation = 0.031, perturbation step size 1 = 0.007, number of iterations K = 10, learning rate 2 = 0.1, batch size m = 128, and run 100 epochs on the training dataset. |