Learning Unforeseen Robustness from Out-of-distribution Data Using Equivariant Domain Translator
Authors: Sicheng Zhu, Bang An, Furong Huang, Sanghyun Hong
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically show the effectiveness of our approach in improving unforeseen and foreseen robustness compared to existing approaches. Additionally, we show that training the equivariant domain translator serves as an effective criterion for source data selection. Fourth, we empirically evaluate our method for image classification tasks on a combination of seven source datasets, two target datasets, and two types of data variations ( 5). We first verify that our method indeed learns equivariant and accurate domain translators. Then, we show the effectiveness of our method in learning unforeseen robustness compared to other baselines, and further support it by ablation studies. |
| Researcher Affiliation | Academia | 1University of Maryland, College Park 2Oregon State University. |
| Pseudocode | Yes | Algorithm 1 Training the domain translator (Py Torch-style pseudocode) Algorithm 2 Training the classifier (Py Torch-style pseudocode) |
| Open Source Code | Yes | We will make our code publicly available at https://github.com/schzhu/unforeseen-robustness. |
| Open Datasets | Yes | Datasets. We use CIFAR-10 and CIFAR-100 (Krizhevsky et al., 2009) as our target datasets, while selecting the source dataset from a range of options including SVHN (Netzer et al., 2011), STL-10 (Coates et al., 2011), CIFAR-100, MNIST (Deng, 2012), Celeb A (Liu et al., 2015), and Caltech-256 (Griffin et al.). |
| Dataset Splits | Yes | Datasets. We use CIFAR-10 and CIFAR-100 (Krizhevsky et al., 2009) as our target datasets... When training domain translators, we only use unlabeled images from the source and target. |
| Hardware Specification | No | The paper mentions "GPU seconds per epoch" in Tables 4 and 5 when discussing computational complexity, but it does not specify any particular GPU models, CPU models, or other hardware specifications used for running the experiments. |
| Software Dependencies | No | The paper mentions "Py Torch-style pseudocode" and the use of "Wasserstein GAN (Arjovsky et al., 2017)", "Res Net18 (He et al., 2016)", "Cycle GAN (Zhu et al., 2017)", and "Kornia" (in footnote 3). However, specific version numbers for PyTorch or other libraries/frameworks are not provided, making it impossible to precisely reproduce the software environment. |
| Experiment Setup | Yes | Other settings. To implement the domain translator, we adopt the encoder-decoder architecture borrowed from Cycle GAN (Zhu et al., 2017), which comprises two downsampling convolutional layers, two residual blocks for latent propagation, and two up-sampling convolutional layers. We use Res Net18 (He et al., 2016) to implement the classifier. We use cross-entropy loss for classification, KL-divergence for consistency regularizing, and mean-squared-error (MSE) loss for measuring the equivariance of the domain translator (the second term in Eq. 4.1). Unless otherwise stated, we set λ=1 in Eq. 4.1 and λ1 =λ2 =0.5 in Eq. 4.2. In this section, we do not apply data augmentation on the source or target to avoid entanglement with the considered data variation. We defer more setup details to Appendix C, and most results on random rotation and CIFAR-100 to Appendix D. |