Understanding new tasks through the lens of training data via exponential tilting
Authors: Subha Maity, Mikhail Yurochkin, Moulinath Banerjee, Yuekai Sun
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the efficacy of our method on WATERBIRDS and BREEDS benchmarks. We demonstrate applications of our method on WATERBIRDS (Sagawa et al., 2019) (Section 5), BREEDS (Santurkar et al., 2020) (Section 6) and synthetic (Appendix C) datasets. |
| Researcher Affiliation | Collaboration | Subha Maity Department of Statistics University of Michigan smaity@umich.edu; Mikhail Yurochkin IBM Research MIT-IBM Watson AI lab mikhail.yurochkin@ibm.com; Moulinath Banerjee Department of Statistics University of Michigan moulib@umich.edu; Yuekai Sun Department of Statistics University of Michigan yuekai@umich.edu |
| Pseudocode | Yes | We summarize the Ex TRA procedure in Algorithm 1 in Appendix B.2. |
| Open Source Code | Yes | 1Codes can be found in https://github.com/smaityumich/exponential-tilting. |
| Open Datasets | Yes | WATERBIRDS dataset combines bird photographs from the Caltech-UCSD Birds-200-2011 (CUB) dataset (Wah et al., 2011) and the image backgrounds from the Places dataset (Zhou et al., 2017). BREEDS (Santurkar et al., 2020) is a subpopulation shift benchmark derived from Image Net (Deng et al., 2009). |
| Dataset Splits | Yes | The source dataset is highly imbalanced, i.e. the smallest group (2) has 56 samples. We consider five subpopulation shift target domains: all pairs of domains with different bird types and the original test set (Sagawa et al., 2019) where all 4 groups are present with proportions vastly different from the source. We evaluate the ability of choosing a model for the target domain based on accuracy on the Ex TRA reweighted source validation data. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU or CPU models, memory, or cloud instance types used for running experiments. |
| Software Dependencies | No | The paper mentions models like ResNet18 and SwAV, and the Adam optimizer, but it does not specify version numbers for any software dependencies, libraries, or frameworks used in the experiments. |
| Experiment Setup | Yes | For both WATERBIRDS and BREEDS datasets, we use Adam optimizer (Kingma & Ba, 2017) with learning rate 1e-4 and batch size 128 for 50 epochs. |