Is Epistemic Uncertainty Faithfully Represented by Evidential Deep Learning Methods?
Authors: Mira Juergens, Nis Meinert, Viktor Bengs, Eyke Hüllermeier, Willem Waegeman
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | This paper presents novel theoretical insights of evidential deep learning, highlighting the difficulties in optimizing second-order loss functions and interpreting the resulting epistemic uncertainty measures. With a systematic setup that covers a wide range of approaches for classification, regression and counts, it provides novel insights into issues of identifiability and convergence in second-order loss minimization, and the relative (rather than absolute) nature of epistemic uncertainty measures. This paper presents novel theoretical results that put previous papers in a unifying perspective by analyzing a broad class of exponential family models that cover classification, regression and count data. For the second research question, we will present in Sections 3.2–3.4 and Section 4 novel theoretical and experimental results. |
| Researcher Affiliation | Academia | 1Department of Data Analysis and Mathematical Modeling, Ghent University, Belgium 2Institue of Communications and Navigation, German Aerospace Center (DLR), Neustrelitz, Germany 3Department of Informatics, University of Munich (LMU), Germany. |
| Pseudocode | No | No pseudocode or algorithm blocks were found within the paper. |
| Open Source Code | Yes | The code for reproducing the experiments is available on Git Hub.2 |
| Open Datasets | No | We generate synthetic training data DN = {(xi, yi)}N i=1 of size N, with xi Unif([0, 0.5]), and yi Bern θ(xi) . The Bernoulli parameter θ is described as a function of the one-dimensional features xi: θ(xi) = 0.5 + 0.4 sin 2πxi, thus θ(xi) [0.5, 0.9]. That is, we generate datasets {(xi, x3 i + ϵ)}N i=1 of different sample sizes, where the instances xi U([ 4, 4]) are uniformly distributed and ϵ N(0, σ2 = 9). |
| Dataset Splits | No | The paper describes generating synthetic training data and resampling it to estimate a reference distribution, but it does not specify explicit training/validation/test splits with percentages, absolute counts, or predefined partition files. |
| Hardware Specification | No | The paper does not specify the exact hardware used for experiments, such as GPU models, CPU types, or memory. |
| Software Dependencies | No | The paper mentions the use of 'Adam optimizer' but does not provide specific version numbers for any software dependencies or libraries (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | We use the same model architecture for first and second-order risk minimization, consisting of 32 neurons and 2 fully-connected hidden layers. The models are trained using the Adam optimizer with a learning rate of 0.0005 and 5000 training epochs. For the regularized models, we use negative entropy of the predicted distribution with λ = 0.01. |