Domain Adaptation under Open Set Label Shift
Authors: Saurabh Garg, Sivaraman Balakrishnan, Zachary Lipton
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments across numerous semi-synthetic benchmarks on vision, language, and medical datasets demonstrate that our methods consistently outperform open set domain adaptation baselines, achieving 10 25% improvements in target domain accuracy. |
| Researcher Affiliation | Academia | Saurabh Garg, Sivaraman Balakrishnan, Zachary C. Lipton Machine Learning Department, Department of Statistics and Data Science, Carnegie Mellon University {sgarg2,sbalakri,zlipton}@andrew.cmu.edu |
| Pseudocode | Yes | Algorithm 1 Positive and Unlabeled learning post Label Shift Estimation (PULSE) framework |
| Open Source Code | Yes | Code is available at https://github.com/acmi-lab/Open-Set-Label-Shift |
| Open Datasets | Yes | For vision, we use CIFAR10, CIFAR100 [40] and Entity30 [61]. For language, we experiment with Newsgroups-20 (http://qwone.com/~jason/20Newsgroups/) dataset. Additionally, inspired by applications of OSLS in biology and medicine, we experiment with Tabula Muris [17] (Gene Ontology prediction), Dermnet (skin disease prediction https://dermnetnz.org/), and Break His [66] (tumor cell classification). |
| Dataset Splits | Yes | We repeat the same process on iid hold out data to obtain validation data with no target labels. |
| Hardware Specification | Yes | All experiments are run on an internal compute cluster with NVIDIA V100 GPUs. |
| Software Dependencies | No | The paper mentions using PyTorch for implementation but does not specify its version or any other software dependencies with version numbers (e.g., 'PyTorch (Paszke et al., 2019)' is a citation, not a version number). |
| Experiment Setup | Yes | We use default hyperparameters for all methods. For OSDA methods, we use default method specific hyperparameters introduced in their works. We provide precise details about hyperparameters, OSLS setup for each dataset and code in App. F.3. |