Causal Balancing for Domain Generalization
Authors: Xinyi Wang, Michael Saxon, Jiachen Li, Hongyang Zhang, Kun Zhang, William Yang Wang
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments are conducted on Domain Bed, demonstrating empirically that our method obtains the best performance across 20 baselines reported on the benchmark. 1 1 INTRODUCTION Our empirical results show that our method obtains significant performance gain compared to 20 baselines on Domain Bed (Arjovsky et al., 2020). 4 EXPERIMENTS |
| Researcher Affiliation | Academia | Xinyi Wang1, Michael Saxon1, Jiachen Li1, Hongyang Zhang2, Kun Zhang3,4, William Yang Wang1 1Department of Computer Science, University of California, Santa Barbara, USA 2David R. Cheriton School of Computer Science, University of Waterloo, Canada 3Department of Philosophy, Carnegie Mellon University, USA 4Machine Learning Department, Mohamed bin Zayed University of Artificial Intelligence, UAE |
| Pseudocode | Yes | Algorithm 1: Balanced Mini-batch sampling. |
| Open Source Code | Yes | 1We publicly release our code at https://github.com/WANGXinyiLinda/causal-balancing-for-domain-generalization. |
| Open Datasets | Yes | Datasets: To verify the effectiveness of our proposed balancing mini-batch method, we conduct experiments on Domain Bed 3, a standard domain generalization benchmark, which contains seven different datasets: Colored MNIST (Arjovsky et al., 2020), Rotated MNIST (Ghifary et al., 2015), VLCS (Fang et al., 2013), PACS (Li et al., 2017), Office Home (Venkateswara et al., 2017), Terra Incognita (Beery et al., 2018) and Domain Net (Peng et al., 2019). 3https://github.com/facebookresearch/DomainBed |
| Dataset Splits | Yes | We primarily consider train domain validation for model selection, as it is the most practical validation method. A detailed description of datasets and baselines, and hyperparameter tuning and selection can be found in Appendix B. Problem Setting. We consider a standard domain generalization setting with a potentially highdimensional variable X (e.g. an image), a label variable Y and a discrete environment (or domain) variable E in the sample spaces X, Y, E, respectively. Here we focus on the classification problems with Y = {1, 2, ..., m} and X Rd. We assume that the training data are collected from a finite subset of training environments Etrain E. The training data De = {(xe i, ye i )}Ne i=1 is then sampled from the distribution pe(X, Y ) = p(X, Y |E = e) for all e Etrain. Our goal is to learn a classifier Cψ : X Y that performs well in a new, unseen environment etest Etrain. Table 1: Out-of-domain accuracy on Colored MNIST10 and Colored MNIST with two train environments [0.1, 0.2] and one test environment [0.9]. |
| Hardware Specification | Yes | All experiments were conducted on NVidia A100, Titan RTX and RTX A6000 GPUs. |
| Software Dependencies | No | The paper mentions using the "Domain Bed codebase", a "multi-layer perceptron (MLP) based VAE", and "Res Net50", but does not specify version numbers for these software components or other libraries (e.g., Python, PyTorch versions). |
| Experiment Setup | Yes | B.3 HYPERPARAMETER SELECTION For Colored MNIST, Colored MNIST10 and Rotated MNIST, we use a 2-layer MLP with 512 neurons in each layer. For all other datasets, we use a 3-layer MLP with 1024 neurons in each layer. We choose the conditional prior pt(Z|Y, E = e) to be a Gaussian distribution with diagonal covariance matrix. We also choose the noise distribution pϵ to be a Gaussian distribution with zero mean and identity variance matrix. Table 3: Choice of hyperparameters for constructing balanced mini-batches, including training the VAE model for latent covariate learning (n, lr, batch size) and the balancing score matching (a, d). |