Robust early-learning: Hindering the memorization of noisy labels
Authors: Xiaobo Xia, Tongliang Liu, Bo Han, Chen Gong, Nannan Wang, Zongyuan Ge, Yi Chang
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on benchmark-simulated and real-world label-noise datasets demonstrate the superiority of the proposed method over the state-of-the-art label-noise learning methods. |
| Researcher Affiliation | Collaboration | 1Trustworthy Machine Learning Lab, School of Computer Science, The University of Sydney 2Department of Computer Science, Hong Kong Baptist University 3School of Computer Science and Engineering, Nanjing University of Science and Technology 4ISN State Key Laboratory, School of Telecommunications Engineering, Xidian University 5Medical AI Group, Faculty of Engineering, Monash University 6Airdoc Research, Monash University 7School of Artificial Intelligence, Jilin University |
| Pseudocode | Yes | Algorithm 1 CDR algorithm. 1: Input: initialization parameters W, noisy training set Dt, noisy validation set Dv, learning rate η, weight decay coefficient λ, fixed τ, epoch T and Tmax, iteration Nmax; |
| Open Source Code | Yes | Our implementation is available at https://github.com/xiaoboxia/CDR. |
| Open Datasets | Yes | To verify the effectiveness of the proposed method, we run experiments on the manually corrupted version of four datasets, i.e., MNIST (Le Cun et al., 1998), F-MNIST (Xiao et al., 2017), CIFAR-10 and CIFAR-100 (Krizhevsky & Hinton, 2009), and two real-world noisy datasets, i.e., Food-101 (Bossard et al., 2014) and Web Vision (Li et al., 2017a). |
| Dataset Splits | Yes | For all datasets, following prior works (Patrini et al., 2017; Wang et al., 2021b), we leave out 10% training data as a validation set, which is for early stopping. |
| Hardware Specification | Yes | For fair comparison, all the codes are implemented in Py Torch 1.2.0 with CUDA 10.0, and run on NVIDIA Tesla V100 GPUs. |
| Software Dependencies | Yes | For fair comparison, all the codes are implemented in Py Torch 1.2.0 with CUDA 10.0, and run on NVIDIA Tesla V100 GPUs. |
| Experiment Setup | Yes | For all the training, we use SGD optimizer with momentum 0.9 and weight decay is set to 10 3. The initial learning rate is set to 10 2. For Food-101, we use a Res Net-50 pre-trained on Image Net with batch size 32. The initial learning rate is changed to 10 3. For Web Vision, we use an Inception-Res Net v2 (Szegedy et al., 2016) with batch size 128. The initial learning rate is set to 10 1. We set 100 epochs in total for all the experiments. |