Learning from Training Dynamics: Identifying Mislabeled Data beyond Manually Designed Features
Authors: Qingrui Jia, Xuhong Li, Lei Yu, Jiang Bian, Penghao Zhao, Shupeng Li, Haoyi Xiong, Dejing Dou
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct extensive experiments to evaluate the proposed method. We train the noise detector based on the synthesized label-noised CIFAR dataset and test such noise detector on Tiny Image Net, CUB-200, Caltech-256, Web Vision and Clothing1M. Results show that the proposed method precisely detects mislabeled samples on various datasets without further adaptation, and outperforms state-of-the-art methods. |
| Researcher Affiliation | Collaboration | 1 Sino-French Engineer School, Beihang University, Beijing, China 2 Baidu Inc. Beijing, China 3 Beihang Hangzhou Innovation Institute Yuhang, Hangzhou, China 4 BCG Greater China, Beijing, China |
| Pseudocode | Yes | Algorithm 1: Supervised Learning from Training Dynamics. |
| Open Source Code | Yes | Code available at https://github.com/Christophe-Jia/mislabeldetection and at Interpret DL (Li et al. 2022) as well. |
| Open Datasets | Yes | Evaluations are conducted on four datasets with synthesized noisy labels: CIFAR10/100 (Krizhevsky and Hinton 2009), CUB-200-2011 (Wah et al. 2011) and Caltech256 (Griffin, Holub, and Perona 2007). ...change the labels of the training set with symmetry (or asymmetry) noises but leave the testing set unchanged. |
| Dataset Splits | No | The paper mentions using 'training set' and 'testing set' for experiments but does not explicitly specify a 'validation set' or its split. |
| Hardware Specification | No | The paper mentions using Res Nets and LSTM models but does not provide specific hardware details such as GPU models, CPU types, or memory specifications used for running experiments. |
| Software Dependencies | No | The paper mentions using Res Nets and an Adam W optimizer but does not specify software libraries with version numbers (e.g., PyTorch 1.x, TensorFlow 2.x, Python 3.x) that were used. |
| Experiment Setup | Yes | Experiment Setups. Evaluations are conducted on four datasets with synthesized noisy labels: CIFAR10/100 (Krizhevsky and Hinton 2009), CUB-200-2011 (Wah et al. 2011) and Caltech256 (Griffin, Holub, and Perona 2007). ...We therefore choose Res Nets as the deep model f ... an LSTM of two hidden layers with 64 dimensions for each is used. ...The optimization process is guided by an Adam W optimizer... |