Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Confidence-Aware Learning for Deep Neural Networks
Authors: Jooyoung Moon, Jihyo Kim, Younghak Shin, Sangheum Hwang
ICML 2020 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experimental results on classification benchmark datasets indicate that the proposed method helps networks to produce well-ranked confidence estimates. |
| Researcher Affiliation | Collaboration | 1Department of Data Science, Seoul National University of Science and Technology, Seoul, Republic of Korea 2LG CNS, Seoul, Republic of Korea 3Department of Industrial & Information Systems Engineering, Seoul National University of Science and Technology, Seoul, Republic of Korea. |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code is available at https://github.com/daintlab/confidenceaware-learning. |
| Open Datasets | Yes | We evaluate our method on benchmark datasets for image classification: SVHN (Netzer et al., 2011) and CIFAR-10/100 (Krizhevsky & Hinton, 2009). |
| Dataset Splits | No | The paper mentions training and testing on benchmark datasets (SVHN, CIFAR-10/100) which have standard splits, but it does not explicitly provide specific percentages, sample counts, or a detailed methodology for a distinct validation split beyond the standard training data used. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., exact GPU/CPU models, memory amounts, or detailed computer specifications) used for running its experiments. |
| Software Dependencies | No | The paper mentions software components and frameworks (e.g., SGD, MCdropout, PyTorch via a GitHub link for a model architecture) but does not provide specific version numbers for any ancillary software dependencies. |
| Experiment Setup | Yes | All models are trained using SGD with a momentum of 0.9, an initial learning rate of 0.1, and a weight decay of 0.0001 for 300 epochs with the mini-batch size of 128. The learning rate is reduced by a factor of 10 at 150 epochs and 250 epochs. |