Domain Generalization via Entropy Regularization
Authors: Shanshan Zhao, Mingming Gong, Tongliang Liu, Huan Fu, Dacheng Tao
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we study domain generalization on four datasets, including two simulated datasets (i.e., Rotated MNIST [11] and Rotated CIFAR-10) and two real-world datasets (i.e., VLCS [11], PACS [28]). We make comparisons against state-of-the-art methods to demonstrate the effectiveness of the proposed algorithm. We conduct extensive ablations to discuss our method comprehensively. |
| Researcher Affiliation | Collaboration | Shanshan Zhao The University of Sydney Australia szha4333@uni.sydney.edu.au Mingming Gong University of Melbourne Australia mingming.gong@unimelb.edu.au Tongliang Liu The University of Sydney Australia tongliang.liu@sydney.edu.au Huan Fu Alibaba Group China fuhuan.fh@alibaba-inc.com Dacheng Tao The University of Sydney Australia dacheng.tao@sydney.edu.au |
| Pseudocode | Yes | Algorithm 1: Training algorithm for domain generalization via entropy regularization. |
| Open Source Code | Yes | Code is available at: https://github.com/sshan-zhao/DG_via_ER. |
| Open Datasets | Yes | Rotated MNIST. Following the setting in [11], we first randomly choose 100 samples per category (1000 in total) from the original dataset [29] to form the domain M0. Rotated CIFAR-10. We randomly choose 500 samples per category (5000 in total) from the original CIFAR-10 dataset [31], and then create additional 5 domains using the same strategy as stated in Rotated MNIST. VLCS. VLCS [11] contains images from four well-known datasets, i.e., Pascal VOC2007 (V) [37], Label Me (L) [38], Caltech (C) [39], and SUN09 (S) [40]. PACS. PACS [28] is proposed specially for domain generalization. It contains four domains, i.e., Photo (P), Art Painting (A), Cartoon (C), and Sketch (S), and seven categories: dog, elephant, giraffe, guitar, house, horse, and person. |
| Dataset Splits | Yes | For VLCS [11]... we randomly split each domain data into training (70%) and test (30%) sets, and do the leave-one-out evaluation. For a fair comparison, we use the same training and validation split as presented in [28]. |
| Hardware Specification | No | The paper mentions using 'Alex Net' and 'Res Net-18' and 'Res Net-50' as backbone networks, which implies the use of GPUs, but no specific GPU model, CPU model, or other hardware specifications are provided. |
| Software Dependencies | No | The paper mentions using 'Adam optimizer' but does not specify software dependencies with version numbers (e.g., Python version, specific deep learning framework versions like PyTorch or TensorFlow). |
| Experiment Setup | Yes | We train our model with the learning rate of 1e 4 (F, T, and D), and 1e 5 ({Ti, T i}5 i=1) for 3, 000 iterations. We set the weighting factors to 0.5 (α1), 0.005 (α2), and 0.01 (α3), respectively. We train the whole network from scratch with the learning rate of 1e 3 (F, T, and D) and 1e 4 ({Ti, T i}5 i=1) using the Adam optimizer [33] for 2000 iterations. The weighting factors (α1, α2, α3) are set to 0.5, 0.001, and 0.1, respectively. We train the model with the batch size of 64 for each source domain for 60 epochs and repeat all of the experiments 20 times. |