Pseudo-Calibration: Improving Predictive Uncertainty Estimation in Unsupervised Domain Adaptation
Authors: Dapeng Hu, Jian Liang, Xinchao Wang, Chuan-Sheng Foo
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive empirical evaluations across 5 diverse UDA scenarios involving 10 UDA methods consistently demonstrate the superior performance and versatility of Pseudo Cal over existing solutions. |
| Researcher Affiliation | Academia | 1Centre for Frontier AI Research, A*STAR, Singapore 2NLPR & MAIS, Institute of Automation, Chinese Academy of Sciences, China 3School of Artificial Intelligence, University of Chinese Academy of Sciences 4National University of Singapore 5Institute for Infocomm Research, A*STAR, Singapore. |
| Pseudocode | Yes | A comprehensive Pytorch-style pseudocode is in Appendix A. |
| Open Source Code | Yes | Code is available at https: //github.com/LHXXHB/Pseudo Cal. |
| Open Datasets | Yes | For image classification, we adopt 5 popular UDA benchmarks of varied scales. Office-31 (Saenko et al., 2010)... Office-Home (Venkateswara et al., 2017)... Vis DA (Peng et al., 2017)... Domain Net (Peng et al., 2019)... Image-Sketch (Wang et al., 2019)... For semantic segmentation, we use Cityscapes(Cordts et al., 2016) as the target domain and either GTA5(Richter et al., 2016) or SYNTHIA (Ros et al., 2016) as the source. |
| Dataset Splits | No | The paper mentions using a 'labeled validation set' in the context of temperature scaling as an existing method, but it does not specify the train/validation/test splits used for its own experimental setup or datasets. |
| Hardware Specification | Yes | We train all UDA models using their official code until convergence on an RTX TITAN GPU. |
| Software Dependencies | No | The paper mentions 'Pytorch-style pseudocode' but does not specify exact version numbers for PyTorch or any other software libraries or dependencies used in the experiments. |
| Experiment Setup | Yes | For Pseudo Cal, a fixed mix ratio λ of 0.65 is employed in all experiments. The UDA model is fixed for only inference use. We use it for one-epoch inference with mixup to generate the labeled pseudo-target set. |