Uncertainty-Aware Deep Classifiers Using Generative Models
Authors: Murat Sensoy, Lance Kaplan, Federico Cerutti, Maryam Saleki5620-5627
AAAI 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Through extensive analysis, we demonstrate that the proposed approach provides better estimates of uncertainty for in- and out-of-distribution samples, and adversarial examples on well-known data sets against state-of-the-art approaches including recent Bayesian approaches for neural networks and anomaly detection methods. |
| Researcher Affiliation | Collaboration | Murat Sensoy,1,2 Lance Kaplan,3 Federico Cerutti,4,5 Maryam Saleki2 1Blue Prism AI Labs, London, UK 2Department of Computer Science, Ozyegin University, Istanbul, Turkey 3US Army Research Lab, Adelphi, MD 20783, USA 4Department of Information Engineering, University of Brescia, 25123 Brescia, Italy 5Cardiff University, Cardiff, CF10 3AT, UK |
| Pseudocode | No | The paper provides mathematical equations and descriptions of the model's components (VAE, GAN, loss functions), but it does not include a block explicitly labeled 'Pseudocode' or 'Algorithm'. |
| Open Source Code | No | The paper mentions: 'We implemented our approach 1 using Python and Tensorflow. 1https://muratsensoy.github.io/gen.html'. However, the linked page states 'Code will be uploaded soon.' and does not provide actual code access for the work described in the paper. |
| Open Datasets | Yes | We implemented our approach 1 using Python and Tensorflow. In this section, we compared our approach with the following approaches: ... evaluated our approach with MNIST and CIFAR10 datasets... We train models for MNIST using the images from 10 digit categories from the training set as usual. However, we then tested these models on not MNIST dataset,4 which contains 10 letters A-J instead of digits. For CIFAR10, we trained models using the training data from the first five categories (referred to as CIFAR5) and tested these models using the images from the last five categories. 4https://www.kaggle.com/lubaroli/notmnist |
| Dataset Splits | No | The paper mentions using training and test sets but does not specify any validation splits or percentages for reproducing the experiments. For example, 'We train models for MNIST using the images from 10 digit categories from the training set as usual.' and 'For CIFAR10, we trained models using the training data from the first five categories (referred to as CIFAR5)'. It does not describe how the training data itself might be split for validation, or if a separate validation set was used for hyperparameter tuning. |
| Hardware Specification | No | The paper does not mention any specific hardware used for running the experiments (e.g., GPU models, CPU types, cloud instances). |
| Software Dependencies | No | The paper mentions 'We implemented our approach 1 using Python and Tensorflow.' (Experimental Section). While it names software, it does not provide specific version numbers for either Python or TensorFlow, which is required for reproducibility. |
| Experiment Setup | Yes | We used Le Net-5 (Le Cun et al. 1998) with Re Lu non-linearities and max pooling as the neural network architecture and evaluated our approach with MNIST and CIFAR10 datasets... We used L2 regularization with coefficient 0.005 in the fully-connected layers. Other approaches are also trained using the same classifier architecture with the priors and posteriors described in (Louizos and Welling 2017) and (Pawlowski et al. 2017). The network architectures in Table 1 to train our model for the MNIST dataset. For CIFAR10, we used the same architectures; however, the classifier uses 192 filters for Conv1 and Conv2, also has 1000 neurons in FC1 as described in (Louizos and Welling 2017). |