Learning Calibrated Uncertainties for Domain Shift: A Distributionally Robust Learning Approach
Authors: Haoxuan Wang, Zhiding Yu, Yisong Yue, Animashree Anandkumar, Anqi Liu, Junchi Yan
IJCAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our method on benchmark datasets and compare our performance with other uncertainty quantification, calibration, and domain adaptation baseline methods. |
| Researcher Affiliation | Collaboration | Haoxuan Wang1 , Zhiding Yu2 , Yisong Yue3 , Animashree Anandkumar3 , Anqi Liu4 and Junchi Yan1 1Department of Computer Science and Engineering and Mo E Key Lab of Artificial Intelligence, Shanghai Jiao Tong University 2NVIDIA 3Department of Computing and Mathematical Sciences, California Institute of Technology 4Department of Computer Science, Johns Hopkins University |
| Pseudocode | Yes | Algorithm 1 End-to-end Training for DRL |
| Open Source Code | Yes | Correspondence authors. Code repository: https://github.com/hatchet Project/Deep Distributionally-Robust-Learning-for-Calibrated-Uncertaintiesunder-Domain-Shift |
| Open Datasets | Yes | We use Office31 [Saenko et al., 2010], Office-Home [Venkateswara et al., 2017] and Vis DA2017 [Peng et al., 2017] for evaluating DRL s uncertainties. We also train models using Image Net [Deng et al., 2009] as the source domain and Image Net V2 [Recht et al., 2019] as the target domain to check the relationship between our estimated weights and the human selection frequencies (HSF) [Chen et al., 2020a]. In addition, we use CIFAR10, STL10 [Coates et al., 2011], MNIST [Lecun and Bottou, 1998] and SVHN [Netzer et al., 2011] to construct cross-domain SSL settings... |
| Dataset Splits | No | The paper mentions using specific datasets (Office31, Office-Home, Vis DA, ImageNet, CIFAR10, STL10, MNIST, SVHN) and training/testing procedures but does not explicitly provide percentages or counts for training, validation, and test splits, nor does it reference predefined splits with citations for reproducibility. |
| Hardware Specification | Yes | All of the training are done on DGX V100 Tesla V100 GPUs with 32GB memory. |
| Software Dependencies | Yes | The main packages and corresponding versions are: Py Torch 0.4.0, CUDA 10.1. |
| Experiment Setup | Yes | For Office31 and Office-Home tasks, we use Res Net50 [He et al., 2016] as the backbone for all models. We train with SGD for 100 epochs and set the learning rate to 0.001. For Vis DA, we use Res Net101 and SGD optimizer. During the 20 epochs of training, the initial learning rate is set as 10 5 and the weight decay parameter is set as 5 10 4. For Image Net, we follow the standard training process of Alex Net [Krizhevsky et al., 2012] and VGG-19 [Simonyan and Zisserman, 2014], where the initial learning rate is 0.01 and we decay the learning rate by a factor of 10 for every 30 epochs. |