Credal Deep Ensembles for Uncertainty Quantification
Authors: Kaizheng Wang, Fabio Cuzzolin, Shireen Kudukkil Manchingal -, Keivan Shariatmadar, David Moens, Hans Hallez
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments are conducted on various out-of-distributions (OOD) detection benchmarks (CIFAR10/100 vs SVHN/Tiny-Image Net, CIFAR10 vs CIFAR10-C, Image Net vs Image Net-O) and using different network architectures (Res Net50, VGG16, and Vi T Base). |
| Researcher Affiliation | Academia | 1KU Leuven, Department of Computer Science, Distri Net 2KU Leuven, Department of Mechanical Engineering, LMSD 3Oxford Brookes University, Visual Artificial Intelligence Laboratory 4Flanders Make@KU Leuven {kaizheng.wang, keivan.shariatmadar, david.moens, hans.hallez}@kuleuven.be {fabio.cuzzolin, 19185895}@brookes.ac.uk |
| Pseudocode | Yes | Algorithm 1 Cre Net Training Procedure |
| Open Source Code | Yes | Codes are available at https://gitlab.kuleuven.be/m-group-campus-brugge/distrinet_public/ credal-deep-ensembles.git. |
| Open Datasets | Yes | Extensive experimental validation is conducted on several OOD detection benchmarks, including CIFAR10/100 (ID) vs SVHN/Tiny-Image Net (OOD), CIFAR10 (ID) vs CIFAR10-C (OOD), Image Net (ID) vs Image Net-O (OOD) |
| Dataset Splits | Yes | After each acquisition step, the model is retrained using the expanded training set. The iterative process continues until either the desired accuracy or the maximum allowable acquired samples are reached. ... After each step, we train models using the Adam optimizer for 20 epochs and select the one with the best accuracy from the validation set. |
| Hardware Specification | Yes | For the main experiments on the Res Net50 backbone, we used two Tesla P100-SXM2-16GB GPUs as devices to independently train 15 SNNs and Cre Nets using CIFAR10 and CIFAR100 datasets. ... In the Image Net experiments, we employed three NVIDIA A100-SXM4-80GB GPUs. ... The time cost is measured on a single Intel Xeon Gold 8358 CPU@2.6 GHz. ... Table 17: Inference cost comparison on CPU between SNNs and Cre Nets per single CIFAR10 input of different architectures. AMD EPYC 7643 48-core CPU. |
| Software Dependencies | No | The paper mentions software components like 'Adam optimizer', 'Sci Py optimization package', and 'Tensor Flow', but does not provide specific version numbers for these dependencies. |
| Experiment Setup | Yes | We trained 15 Cre Nets (using δ =0.5) and SNNs on the Res Net50 architecture [28] starting from different random seeds... We employed the Adam optimizer, with a learning rate scheduler set at 0.001 and reduced to 0.0001 during the last five training epochs. ... VGG16-based SNNs and Cre Nets were trained for 20 epochs. SNNs and Cre Nets using the Vi T Base backbone were trained for 25 and 40 epochs, respectively. |