Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptation
Authors: Kendrick Shen, Robbie M Jones, Ananya Kumar, Sang Michael Xie, Jeff Z. Haochen, Tengyu Ma, Percy Liang
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically validate our theory on benchmark vision datasets.In our experiments, contrastive pre-training obtains comparable or better results to strong UDA methods. |
| Researcher Affiliation | Academia | 1Department of Computer Science, Stanford University. Correspondence to: Kendrick Shen, Robbie Jones, Ananya Kumar, Sang Michael Xie <{kshen6, rmjones, ananya, xie}@cs.stanford.edu>. |
| Pseudocode | No | No structured pseudocode or algorithm blocks were found. |
| Open Source Code | No | The paper mentions using 'the authors official repository (https://github.com/virajprabhu/SENTRY)' for SENTRY, 'the official Sw AV implementation (https://github.com/facebookresearch/swav)' for Sw AV, and 'the official Sim CLR implementation (https://github.com/google-research/simclr)' for Sim CLR. These refer to existing implementations of baseline or compared methods, not the specific code for the methodology described in this paper by the current authors. |
| Open Datasets | Yes | We conduct experiments on Domain Net (Peng et al., 2019; Prabhu et al., 2021), which contains 40 classes and 4 domains, BREEDS Living-17 and Entity-30 (Santurkar et al., 2020), which are adaptation benchmarks derived from Image Net, and STL-10 CIFAR-10 (Coates et al., 2011; Krizhevsky, 2009; French et al., 2018), which are two classical image recognition datasets often paired together for domain adaptation. |
| Dataset Splits | No | The paper describes the use of labeled source data and unlabeled source/target data, and mentions fine-tuning on labeled source data and evaluating on target data. It refers to standard datasets like Domain Net and BREEDS, which have pre-defined splits, but does not explicitly state the specific train/validation/test percentages or sample counts within the paper's text. |
| Hardware Specification | No | No specific hardware details (such as GPU/CPU models, processor types, or memory amounts) used for running the experiments were provided. |
| Software Dependencies | No | The paper mentions software components such as 'Py Torch torchvision library' and algorithms like 'Sw AV', 'Sim CLR', 'SENTRY', 'DANN', but does not provide specific version numbers for these or other underlying software dependencies (e.g., Python, PyTorch, CUDA). |
| Experiment Setup | Yes | For each pair of domains, we conduct a hyperparameter search through λsrc {0.5,1.0,1.5} (the weight on the supervised classification loss) and learning rates {0.01,0.001}. We run all hyperparameter settings for 100 epochs from a 150-epoch ERM checkpoint and select the best one (based on target test accuracy). We set the number of prototypes to be 10 times the number of classes (170, 300, and 400 for Living-17, Entity-30 and Domain Net, respectively). We set ϵ=0.03. We set the base learning rate to 0.6, following a linear scaling rule based on batch size. |