Confidence-aware Contrastive Learning for Selective Classification
Authors: Yu-Chang Wu, Shen-Huan Lyu, Haopu Shang, Xiangyu Wang, Chao Qian
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The experimental results on typical datasets, i.e., CIFAR-10, CIFAR100, Celeb A, and Image Net, show that CCL-SC achieves significantly lower selective risk than state-of-the-art methods, across almost all coverage degrees. |
| Researcher Affiliation | Academia | 1National Key Laboratory for Novel Software Technology, Nanjing University, China 2 School of Artificial Intelligence, Nanjing University, China 3Key Laboratory of Water Big Data Technology of Ministry of Water Resources, Hohai University, China 4College of Computer Science and Software Engineering, Hohai University, China. |
| Pseudocode | Yes | The pseudo-code of the training method is shown in Algorithm 1 in Appendix C. |
| Open Source Code | Yes | The codes are provided in https: //github.com/lamda-bbo/CCL-SC. |
| Open Datasets | Yes | We conduct experiments on four commonly used datasets, i.e., Celeb A (Liu et al., 2015), CIFAR-10/CIFAR100 (Krizhevsky et al., 2009), and Image Net (Deng et al., 2009). |
| Dataset Splits | Yes | For each dataset, we utilize 20% of the training set as the validation set to tune hyper-parameters. |
| Hardware Specification | No | This paper does not explicitly state the specific hardware (e.g., GPU models, CPU models, or memory) used for running experiments. |
| Software Dependencies | No | This paper does not explicitly provide specific version numbers for software dependencies like deep learning frameworks or libraries. |
| Experiment Setup | Yes | Hyper-parameters For each dataset, we utilize 20% of the training set as the validation set to tune hyper-parameters. We test the momentum coefficient q {0.9, 0.999, 0.999}, and the weight coefficient w {0.1, 0.5, 1.0}. For the queue size s, we set it based on the number of classes in the dataset: for datasets with fewer classes such as Celeb A and CIFAR-10, s = 300; for datasets with more classes such as CIFAR-100 and Image Net, we set s = 3000 and s = 10000, respectively. We train the model on the entire training set to evaluate performance. Detailed hyperparameter settings for each method on each dataset are provided in Appendix D.1. |