Adaptive Uncertainty Estimation via High-Dimensional Testing on Latent Representations
Authors: Tsai Hor Chan, Kin Wai Lau, Jiajun Shen, Guosheng Yin, Lequan Yu
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments validate the satisfactory performance of our framework on uncertainty estimation and task-specific prediction over a variety of competitors. The experiments on the OOD detection task also show satisfactory performance of our method when the OOD data are unseen in the training. |
| Researcher Affiliation | Collaboration | Tsai Hor Chan Department of Statistics and Actuarial Science The University of Hong Kong hchanth@connect.hku.hk Kin Wai Lau TCL AI Lab Hong Kong stevenlau@tcl.com Jiajun Shen TCL AI Lab Hong Kong shenjiajun90@gmail.com Guosheng Yin Department of Mathematics Imperial College London guosheng.yin@imperial.ac.uk Lequan Yu Department of Statistics and Actuarial Science The University of Hong Kong lqyu@hku.hk |
| Pseudocode | Yes | Algorithm 1 gives the detailed workflow of our proposed uncertainty estimation framework. |
| Open Source Code | Yes | Codes are available at https://github.com/HKU-Med AI/bnn_uncertainty. |
| Open Datasets | Yes | For the OOD detection task, we treat CIFAR 10 and MNIST as the in-distribution datasets, and Fashion MNIST, OMNIGLOT, and SVHN as the OOD datasets. To validate the advantage of BNN encoding on limited observations, we compose a medical image benchmark using samples from the Diabetes Retinopathy Detection (DRD) dataset [8], with samples shown in Figure 3. We treat healthy samples as in-distribution data and unhealthy samples as OOD data. |
| Dataset Splits | Yes | Five-fold cross-validation is applied to each of the competitive methods. |
| Hardware Specification | Yes | The proposed method is implemented in Python with Pytorch library on a server equipped with four NVIDIA TESLA V100 GPUs. |
| Software Dependencies | No | The paper states 'The proposed method is implemented in Python with Pytorch library' but does not provide specific version numbers for these software components or any other libraries used. |
| Experiment Setup | Yes | The dropout ratio of each dropout layer is selected as 0.2. All models are trained with 100 epochs with possible early stopping. We use the Adam optimizer to optimize the model with a learning rate of 5 × 10−5 and a weight decay of 1 × 10−5. Data augmentations such as color jittering and random cropping and flipping are applied as a regularization measure. Prior mean of weights sampled from N(−3, 0.01). n2 = 300 λ0 = 0.01 |