Coupled Confusion Correction: Learning from Crowds with Sparse Annotations
Authors: Hansong Zhang, Shikun Li, Dan Zeng, Chenggang Yan, Shiming Ge
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments are conducted on two types of synthetic datasets and three real-world datasets, the results of which demonstrate that CCC significantly outperforms state-of-the-art approaches. |
| Researcher Affiliation | Academia | 1Institute of Information Engineering, Chinese Academy of Sciences, Beijing 100092, China 2School of Cyber Security, University of Chinese Academy of Sciences, Beijing 100049, China 3Department of Communication Engineering, Shanghai University, Shanghai 200040, China 4Hangzhou Dianzi University, Hangzhou 310018, China |
| Pseudocode | Yes | the pseudo code can be found in the Appendix. |
| Open Source Code | Yes | Source codes are available at: https://github.com/Hansong-Zhang/CCC. |
| Open Datasets | Yes | We evaluate our method on a variety of synthetic datasets, which are derived from two widely used datasets, i.e., CIFAR-10 (Krizhevsky, Hinton et al. 2009) and Fashion-MNIST (Xiao, Rasul, and Vollgraf 2017). ... Three real-world datasets are adopted for evaluation, including Label Me (Rodrigues and Pereira 2018), CIFAR-10N (Wei et al. 2022b) and MUSIC (Rodrigues, Pereira, and Ribeiro 2014). |
| Dataset Splits | Yes | In the meta-learning-based methods (Shu et al. 2019; Wang, Hu, and Hu 2020; Zheng, Awadallah, and Dumais 2021; Xu et al. 2021; Zhang and Pfister 2021), an out-of-sample meta set Dmeta = {xmeta j , ymeta j }M j=1 is assumed to be available... All experiments are repeated three times with different random seeds, both the best accuracy on test set during training and the accuracy at the last epoch is reported. |
| Hardware Specification | Yes | All experiments are conducted on a single NVIDIA RTX 3090 GPU. |
| Software Dependencies | No | The paper mentions using PyTorch but does not specify the version number or other software dependencies with their versions in the provided text. |
| Experiment Setup | Yes | All experiments are repeated three times with different random seeds, both the best accuracy on test set during training and the accuracy at the last epoch is reported. ... Let ηv, ηm, ηa denote the learning rates in the Virtual, Meta, and Actual stages respectively. ... a training mini-batch M = {xi, yi}n i=1 and a meta mini-batch Mmeta = {xmeta j , ymeta j }m j=1 is fetched, where the n and m is the batch size of training and meta set respectively. ... more information about datasets and the implementation details can be found in the Appendix. |