Uncertainty-Aware Learning against Label Noise on Imbalanced Datasets
Authors: Yingsong Huang, Bing Bai, Shengwei Zhao, Kun Bai, Fei Wang6960-6969
AAAI 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct experiments on several synthetic and real-world datasets. The results demonstrate the effectiveness of the proposed method, especially on imbalanced datasets.In this section, we report the experimental results. We introduce the experimental setup, report the performance of ULC and baselines, and also analyze the results of ablation experiments. |
| Researcher Affiliation | Collaboration | Yingsong Huang1*, Bing Bai1*, Shengwei Zhao1, Kun Bai1, Fei Wang2 1Tencent Security Big Data Lab, Tencent Inc., China 2Department of Population Health Sciences, Weill Cornell Medicine, USA |
| Pseudocode | No | The paper outlines the algorithm procedure in text but does not contain a formally structured 'Pseudocode' or 'Algorithm' block. |
| Open Source Code | No | The paper does not contain any explicit statement about releasing the source code or a link to a code repository for the methodology described. |
| Open Datasets | Yes | We perform extensive evaluations on five datasets: CIFAR-10, CIFAR-100, class-imbalanced CIFAR10, class-imbalanced CIFAR-100, and Clothing1M. The noise injection methods for CIFAR-10 and CIFAR-100 follow the previous work (Li, Socher, and Hoi 2020). |
| Dataset Splits | Yes | Note that we only resample the training sets for class imbalance settings, i.e., the test sets remain the same as class balance settings. Then, labels in class-imbalanced CIFAR get flipped to the rest of the categories with the same probability. Clothing1M is a large-scale dataset with real-world noisy labels. In our experiment, clean training data is not used. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., exact GPU/CPU models or processor types) used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific software dependency details with version numbers (e.g., library or solver names with version numbers) needed to replicate the experiment. |
| Experiment Setup | Yes | For CIFAR experiments, we use the Pre Act Res Net-18 (He et al. 2016) which is commonly used to benchmark label noise learning methods (Li, Socher, and Hoi 2020). We train the network using SGD with a momentum of 0.9 for 300 epochs; warm-up 30 epochs for CIFAR-10 and 40 epochs for CIFAR-100. In the Clothing1M experiments, we use Res Net-50 with Image Net pretrained weights, following the previous work (Li, Socher, and Hoi 2020). The warm-up period is 1 epoch for Clothing1M. τ is set as 0.6 for 90% noise ratio and 0.5 for others. λu is validated from {0, 25, 50, 150}. Generally, the hyperparameters setting for Mix Match is inherited from Divide Mix without heavily tuning, because the SSL part is not our focus and can be replaced by other alternatives. We leverage MC-dropout (Gal and Ghahramani 2016) to estimate uncertainty, setting T to 10 and the dropout rate to 0.3. The uncertainty ratio r is set as 0.1 to obtain the final clean probability. We model aleatoric uncertainty for classdependent and instance-dependent noise by 10 Monte Carlo samples on logits. |