Combating Label Noise in Deep Learning using Abstention
Authors: Sunil Thulasidasan, Tanmoy Bhattacharya, Jeff Bilmes, Gopinath Chennupati, Jamal Mohd-Yusof
ICML 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the utility of the deep abstaining classifier for various image classification tasks under different types of label noise; in the case of arbitrary label noise, we show significant improvements over previously published results on multiple image benchmarks. |
| Researcher Affiliation | Collaboration | 1Los Alamos National Laboratory, Los Alamos, NM, USA 2Department of Electrical & Computer Engineering, University of Washington, Seattle, WA, USA. |
| Pseudocode | Yes | Algorithm 1 α auto-tuning |
| Open Source Code | No | The paper does not explicitly state that source code for the described methodology is made openly available, nor does it provide a link to a code repository. It only mentions the use of PyTorch framework. |
| Open Datasets | Yes | We use the labeled version of the STL-10 dataset (Coates et al., 2011), comprising of 5000 and 8000 96x96 RGB images in the train and test set respectively, augmented with random crops and horizontal flips during training. |
| Dataset Splits | No | The paper mentions using a 'validation set' to track loss and determine the best performing epoch. However, it does not provide specific details on the size, percentage, or methodology for splitting the dataset into training, validation, and test sets, nor does it cite predefined splits for the validation set for the CIFAR/Fashion-MNIST datasets. For STL-10, only train and test sizes are explicitly given, with no separate validation set mentioned. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running experiments, such as GPU models, CPU specifications, or cloud computing resources. |
| Software Dependencies | No | The paper mentions that the architecture was 'implemented in the Py Torch (Paszke et al., 2017) framework' but does not specify a version number for PyTorch or any other software dependencies. |
| Experiment Setup | Yes | Setup: For the experiments in this section, we use a deep convolutional network employing the VGG-16 (Simonyan & Zisserman, 2014) architecture, implemented in the Py Torch (Paszke et al., 2017) framework. We train the network for 200 epochs using SGD accelerated with Nesterov momentum and employ a weight decay of .0005, initial learning rate of 0.1 and learning rate annealing using an annealing factor of 0.5 at epoch 60, 120 and 160. We perform abstention-free training during the first 20 epochs... For the α autoupdate algorithm we set ρ (α initialization factor) to 64 and µ to 0.05; we did not tune these parameters. |