Free Lunch for Domain Adversarial Training: Environment Label Smoothing
Authors: YiFan Zhang, xue wang, Jian Liang, Zhang Zhang, Liang Wang, Rong Jin, Tieniu Tan
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate, both experimentally and theoretically, that ELS can improve training stability, local convergence, and robustness to noisy environment labels. |
| Researcher Affiliation | Collaboration | Yi-Fan Zhang1,2 , Xue Wang3, Jian Liang1,2, Zhang Zhang1,2, Liang Wang1,2, Rong Jin3 , Tieniu Tan1,2 1National Laboratory of Pattern Recognition (NLPR), Institute of Automation 2School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS) 3 Machine Intelligence Technology, Alibaba Group. |
| Pseudocode | No | The paper does not include any pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | Yes | The code is avaliable at https://github.com/yfzhang114/Environment-Label-Smoothing. |
| Open Datasets | Yes | Rotated MNIST (Ghifary et al., 2015) consists of 70,000 digits in MNIST with different rotated angles where domain is determined by the degrees d {0,15,30,45,60,75}. PACS (Li et al., 2017b) includes 9, 991 images with 7 classes [...] from 4 domains. VLCS (Torralba & Efros, 2011) is composed of 10,729 images, 5 classes [...] from domains. Office-31 (Saenko et al., 2010) contains contains 4,110 images, 31 object categories in three domains. Office-Home (Venkateswara et al., 2017): consists of 15,500 images from 65 classes and 4 domains. ... Civil Comments-Wilds (Koh et al., 2021) contains 448,000 comments... Amazon-Wilds (Koh et al., 2021) contains 539,520 reviews... Rx Rx1-wilds (Koh et al., 2021) comprises images of cells... OGB-Mol PCBA (Koh et al., 2021) is a multi-label classification dataset... Spurious-Fourier (Gagnon-Audet et al., 2022) is a binary classification dataset... HHAR (Gagnon-Audet et al., 2022) is a 6 activities classification dataset... |
| Dataset Splits | Yes | The model selection that we use is test-domain validation, one of the three selection methods in (Gulrajani & Lopez-Paz, 2021). That is, we choose the model maximizing the accuracy on a validation set that follows the same distribution of the test domain. For DA tasks, all baselines implementation and hyper-parameters follows (Wang & Hou). |
| Hardware Specification | Yes | We conduct all the experiments on a machine with i7-8700K, 32G RAM, and four GTX2080ti. |
| Software Dependencies | No | The paper mentions software like 'Py Torch' and 'Distill BERT', but it does not provide specific version numbers for these software dependencies, which is required for reproducibility. |
| Experiment Setup | Yes | Table 9: Hyper-parameters for different benchmarks. Lrg,Decayg: learning rate and weight decay for the encoder and classifier; Lrd,Decayd: learning rate and weight decay for the domain discriminator; bsz: batch size during training; dsteps: the discriminator is trained dsteps times once the encoder and classifier are trained; Wreg: tradeoff weight for the gradient penalty; λ: tradeoff weight for the adversarial loss. The default β2 for Adam and Adam W optimizer is 0.99 and the momentum for SGD optimizer is 0.9. |