AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation
Authors: David Berthelot, Rebecca Roelofs, Kihyuk Sohn, Nicholas Carlini, Alexey Kurakin
ICLR 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We extend semi-supervised learning to the problem of domain adaptation to learn significantly higher-accuracy models that train on one data distribution and test on a different one. With the goal of generality, we introduce Ada Match, a unified solution for unsupervised domain adaptation (UDA), semi-supervised learning (SSL), and semi-supervised domain adaptation (SSDA). In an extensive experimental study, we compare its behavior with respective state-of-the-art techniques from SSL, SSDA, and UDA and find that Ada Match either matches or significantly exceeds the state-of-the-art in each case using the same hyper-parameters regardless of the dataset or task. |
| Researcher Affiliation | Industry | David Berthelot , Rebecca Roelofs , Kihyuk Sohn , Nicholas Carlini , Alex Kurakin Google Research |
| Pseudocode | No | The paper describes the method and its components in detail with mathematical equations, but it does not include a figure, block, or section explicitly labeled "Pseudocode" or "Algorithm". |
| Open Source Code | Yes | 1Code to reproduce results: https://github.com/google-research/adamatch |
| Open Datasets | Yes | We evaluate Ada Match on the SSL, UDA, and SSDA tasks using the standard Digit Five (Ganin et al., 2016) and Domain Net (Peng et al., 2019) visual domain adaptation benchmarks |
| Dataset Splits | No | In the SSL setting, we only train on a single dataset (since there is no notion of source nor target) which we randomly divide into two groups: labeled and unlabeled. The paper does not provide explicit training/validation/test dataset splits with percentages, absolute sample counts, or citations to predefined splits for general reproducibility. |
| Hardware Specification | Yes | Digit Five experiments and ablation studies were run on a single V100 GPU, other experiments were run on a single TPU. |
| Software Dependencies | No | We wrote our experimental codes using the open-source Objax library (Objax Developers, 2020) and used the same hyperparameters for most of our experiments, which we specify in Section 4. The paper mentions the 'Objax library' but does not specify its version number or any other software dependencies with version numbers. |
| Experiment Setup | Yes | Network and training hyperparameters. We use Res Net V2-101 (He et al., 2016) for resolution 224 224 , WRN-34-2 (Zagoruyko & Komodakis, 2016) for 64 64, and WRN-28-2 for 32 32. We set learning rate to 0.03 and learning rate cosine decay to 0.25. We trained Digit Five for 32M images, and Domain Net for 8M. Ada Match only requires the following two hyper-parameters: (1) Confidence threshold τ (set to 0.9 for all experiments). (2) Unlabeled target batch size ratio uratio (set to 3 for all experiments) which defines how much larger is the unlabeled batch, e.g. ntu = nsl uratio. |