Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
FINE Samples for Learning with Noisy Labels
Authors: Taehyeon Kim, Jongwoo Ko, sangwook Cho, JinHwan Choi, Se-Young Yun
NeurIPS 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results show that the proposed methods consistently outperform corresponding baselines for all three applications on various benchmark datasets. 1 Introduction Deep neural networks (DNNs) have achieved remarkable success in numerous tasks as the amount of accessible data has dramatically increased [21, 15]. |
| Researcher Affiliation | Academia | Taehyeon Kim KAIST AI KAIST Daejeon, South Korea EMAIL Jongwoo Ko KAIST AI KAIST Daejeon, South Korea EMAIL Sangwook Cho KAIST AI KAIST Daejeon, South Korea EMAIL Jinhwan Choi KAIST AI KAIST Daejeon, South Korea EMAIL Se-Young Yun KAIST AI KAIST Daejeon, South Korea EMAIL |
| Pseudocode | Yes | Algorithm 1: FINE Algorithm for Sample Selection |
| Open Source Code | Yes | Code available at https://github.com/Kthyeon/FINE_official |
| Open Datasets | Yes | Noisy Benchmark Dataset. Following the previous setups [25, 29], we artificially generate two types of random noisy labels: injecting uniform randomness into a fraction of labels (symmetric) and corrupting a label only to a specific class (asymmetric). For example, we generate noise by mapping TRUCK AUTOMOBILE, BIRD AIRPLANE, DEER HORSE, CAT DOG to make asymmetric noise for CIFAR-10. For CIFAR-100, we create 20 five-size super-classes and generate asymmetric noise by changing each class to the next class within super-classes circularly. For a real-world dataset, Clothing1M [44] containing inherent noisy labels is used. |
| Dataset Splits | Yes | The dataset provides 50k, 14k, and 10k verified as clean data for training, validation, and testing. |
| Hardware Specification | No | No specific hardware details (e.g., GPU/CPU models, memory) are mentioned for the experiments. |
| Software Dependencies | No | No specific ancillary software details with version numbers (e.g., PyTorch 1.9, Python 3.8) are provided. |
| Experiment Setup | Yes | We set the threshold ζ as 0.5. |