A Theory of Label Propagation for Subpopulation Shift
Authors: Tianle Cai, Ruiqi Gao, Jason Lee, Qi Lei
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We implement the popular consistency-based semi-supervised learning algorithm Fix Match (Sohn et al., 2020) on the subpopulation shift task from BREEDS (Santurkar et al., 2021), and compare it with popular distributional matching methods (Ganin et al., 2016; Zhang et al., 2019). Results show that the consistency-based method outperforms distributional matching methods by over 8%, partially verifying our theory on the subpopulation shift problem. We also show that combining distributional matching methods and consistency-based algorithm can improve the performance upon distributional matching methods on classic unsupervised domain adaptation datasets such as Office31 (Saenko et al., 2010) and Office-Home (Venkateswara et al., 2017). |
| Researcher Affiliation | Academia | 1Princeton University 2Zhongguancun Haihua Institute for Frontier Information Technology. Correspondence to: Jason D. Lee <jasonlee@princeton.edu>. |
| Pseudocode | No | The paper describes the algorithm (e.g., equation (1) and (2)) in textual and mathematical form but does not include structured pseudocode or an algorithm block. |
| Open Source Code | No | We use the implementation from Junguang Jiang (2020), which shows that MDD has the best performance among the evaluated methods. |
| Open Datasets | Yes | We conduct experiments on a dataset that is constructed to simulate natural subpopulation shift. Towards this goal, we constructed an Unsupervised Domain Adaptation (UDA) task using the challenging ENTITY-30 task from BREEDS tasks (Santurkar et al., 2021)... |
| Dataset Splits | No | The paper mentions training on source and evaluating on target domains but does not explicitly provide details about train/validation/test dataset splits (e.g., percentages or sample counts) in the provided text. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware used for running the experiments (e.g., GPU models, CPU types, or memory). |
| Software Dependencies | No | The paper mentions using specific algorithms and libraries like 'Fix Match', 'MDD', and 'SwAV', and references the 'Transferlearning-library', but does not provide specific version numbers for any software dependencies. |
| Experiment Setup | No | We defer the detailed experimental settings to Appendix C and report the results here. |