Finding Actual Descent Directions for Adversarial Training
Authors: Fabian Latorre, Igor Krawczuk, Leello Tadesse Dadi, Thomas Pethick, Volkan Cevher
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In Section 5 we verify experimentally that: (i) it is unrealistic to assume a unique solution of the inner-maximization problem, hence making a case for our method DDi, (ii) our method can achieve more stable descent dynamics than the vanilla AT method in synthetic scenarions and (iii) on the CIFAR10 dataset DDi is more stable and achieves higher robustness levels in the early stages of traning, compared with a PGD adversary of equivalent complexity. |
| Researcher Affiliation | Academia | Fabian Latorre , Igor Krawczuk , Leello Dadi , Thomas Pethick and Volkan Cevher EPFL, Switzerland firstname.lastname@epfl.ch |
| Pseudocode | Yes | Algorithm 1 Danskin s Descent Direction (DDi) |
| Open Source Code | No | The code to reproduce our results will be available at https://github.com/LIONS-EPFL/ddi_at. |
| Open Datasets | Yes | Using the CIFAR10 dataset we further provide a real world example showing that our method achieves a steeper increase in robustness levels in the early training stages of smooth-activation networks without Batch Norm, and is more stable than the PGD baseline. |
| Dataset Splits | Yes | Figure 4: (left) Evolution of the robust accuracy on the CIFAR10 validation set, using a standard PGD-20 adversary for evaluation and DDi/PGD-7 during training. |
| Hardware Specification | No | The paper does not specify any particular hardware (e.g., GPU, CPU models, memory specifications) used for conducting the experiments. |
| Software Dependencies | No | The paper mentions software components and algorithms such as PGD, SGD, CELU, Batch Norm, and Group Norm but does not provide specific version numbers for any software dependencies. |
| Experiment Setup | Yes | This means SGD with hyperparameters lr= 0.1, momentum=0.0 (not the default 0.9, we explain why below), batch size= 128 and weight decay= 5e 4. We run for 200 epochs, no warmup, decreasing lr by a factor of 0.1 at 50% and 75% of the epochs. |