Adversarial Self-Training Improves Robustness and Generalization for Gradual Domain Adaptation
Authors: Lianghe Shi, Weiwei Liu
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct comprehensive experiments to validate the effectiveness of gradual AST in Section 3. Compared to the gradual self-training method, the proposed gradual AST method delivers a great improvement in adversarial accuracy, from 6.00% to 90.44% on the Rotating MNIST dataset. More interestingly, we find that the clean accuracy of gradual AST on the target domain also increases from 90.06% to 97.15%. |
| Researcher Affiliation | Academia | Lianghe Shi Weiwei Liu School of Computer Science, Wuhan University National Engineering Research Center for Multimedia Software, Wuhan University Institute of Artificial Intelligence, Wuhan University Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University |
| Pseudocode | No | The paper describes methods using mathematical formulations and textual descriptions but does not include any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code is available at https://github.com/whustone007/AST_GDA. |
| Open Datasets | Yes | Rotating MNIST is a semi-synthetic dataset generated by rotating each MNIST image at an angle between 0 and 60 degrees. The 50,000 training set images are divided into three parts: a source domain of 5000 images (0-5 degrees), 21 intermediate domains of 42,000 images (5-60 degrees), and a set of validation data. The rotating degree gradually increases along the domain sequence. The 10,000 test set images are rotated by 55-60 degrees, representing the target domain. Portraits [18] is a real dataset consisting portraits of American high school students across a century. |
| Dataset Splits | Yes | The 50,000 training set images are divided into three parts: a source domain of 5000 images (0-5 degrees), 21 intermediate domains of 42,000 images (5-60 degrees), and a set of validation data. The 10,000 test set images are rotated by 55-60 degrees, representing the target domain. |
| Hardware Specification | Yes | We implement our methods using Py Torch [37] on two Nvidia Ge Force RTX 3090 Ti GPUs. |
| Software Dependencies | No | We implement our methods using Py Torch [37] on two Nvidia Ge Force RTX 3090 Ti GPUs. (Only PyTorch is mentioned without a specific version number.) |
| Experiment Setup | Yes | Following [26], we use a 3-layer convolutional network with dropout (0.5) and Batch Norm on the last layer. We use mini-batch stochastic gradient descent (SGD) with momentum 0.9 and weight decay 0.001. The batch size is 32 and the learning rate is 0.001. We train the model for 40 epochs in each domain. We set the radius of the bounded perturbation to be 0.1 [51] for Rotating MNIST and 0.031 for Portraits. Following [33], we use PGD-20 with a single step size of 0.01 as the adversarial attacker. |