Single-Model Uncertainties for Deep Learning
Authors: Natasa Tagasovska, David Lopez-Paz
NeurIPS 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To this end, we contribute: Simultaneous Quantile Regression (SQR) to estimate aleatoric uncertainty (Section 2), Orthonormal Certificates (OCs) to estimate epistemic uncertainty (Section 3), experiments showing the competitive performance of these estimators (Section 4), and an unified literature review on uncertainty estimation (Section 5). |
| Researcher Affiliation | Collaboration | Natasa Tagasovska Department of Information Systems HEC Lausanne, Switzerland natasa.tagasovska@unil.ch David Lopez-Paz Facebook AI Research Paris, France dlp@fb.com |
| Pseudocode | No | The paper includes mathematical formulations and descriptions of algorithms but does not contain a clearly labeled 'Pseudocode' or 'Algorithm' block. |
| Open Source Code | Yes | Our code is available at https://github.com/facebookresearch/Single_Model_Uncertainty. |
| Open Datasets | Yes | We evaluate SQR (2) to construct (1 α) Prediction Intervals (PIs) on eight UCI datasets [4]. We consider four classification datasets with ten classes: MNIST, CIFAR-10, Fashion-MNIST, and SVHN. |
| Dataset Splits | Yes | This table shows test average and standard deviation PICP of those models achieving a validation PICP in [0.925, 0.975]. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running experiments (e.g., GPU models, CPU specifications, or memory details). |
| Software Dependencies | No | The paper mentions using the 'Adam optimizer [46]' and neural network architectures like 'Pre Act Res Net18 [34, 51] and VGG [71]', but it does not specify software versions for programming languages, libraries, or frameworks (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | We use the same neural network architecture for the first three methods, and cross-validate the learning rate and weight decay parameters for the Adam optimizer [46]. For the tree-based of the methods, we cross-validate the number of trees and the minimum number of examples to make a split. We repeat all experiments across 20 random seeds. See Appendix C and code for details. |