Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Re-Evaluating the Impact of Unseen-Class Unlabeled Data on Semi-Supervised Learning Model
Authors: Rundong He, Yicong Dong, Lan-Zhe Guo, Yilong Yin, Tailin Wu
ICLR 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments demonstrate that unseen classes in unlabeled data do not necessarily impair the performance of SSL models; in fact, under certain conditions, unseen classes may even enhance them. Table 1 and 2 presents the evaluation of various SSL methods on CIFAR10 and CIFAR100, with assessments conducted using 100 labels within inconsistent label spaces and 1000 labels within inconsistent label spaces, respectively. |
| Researcher Affiliation | Academia | 1 Department of Artificial Intelligence, School of Engineering, Westlake University 2 School of Software, Shandong University 3 School of Intelligence Science and Technology, Nanjing University rundong EMAIL, EMAIL, EMAIL |
| Pseudocode | No | The paper describes methodologies and evaluation frameworks but does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code is available at https://github.com/rundonghe/RESSL. |
| Open Datasets | Yes | Table 1 and 2 presents the evaluation of various SSL methods on CIFAR10 and CIFAR100... To investigate the impact of nearness on SSL models, we have also conducted experiments where the unseen classes come from MNIST... Evaluation on Image Net-100... Evaluation on CUB... Evaluation on Forest under inconsistent label spaces... Evaluation on Letter under inconsistent label spaces... Evaluation on AGNews under inconsistent label spaces. |
| Dataset Splits | Yes | For CIFAR-10, we select the first 5 classes as seen classes and the last 5 classes as unseen classes. From the training set of known classes (which contains 25,000 samples), we randomly select 100 samples to form the labeled set DL. We then randomly select rs 25,000 samples from the remaining samples of the training set of known classes and put them into DS U. From the training set of unknown classes (which contains 25,000 samples), we randomly select ru 25,000 samples and put them into DU U . DS U and DU U together form the unlabeled set DU. |
| Hardware Specification | Yes | All experiments of deep SSL algorithms are conducted with 4 NVIDIA RTX A6000 GPUs. |
| Software Dependencies | No | The paper mentions 'LAMDA-SSL toolkit' but does not specify its version number or versions of other software dependencies. |
| Experiment Setup | Yes | For all the experiments, we use supervised learning algorithm as baselines, which trains model based on DL with empirical risk minimization. The evaluation of the algorithm was performed based on the LAMDA-SSL toolkit Jia et al. (2022). We use Res Net50 He et al. (2016)as the backbone. All experiments of deep SSL algorithms are conducted with 4 NVIDIA RTX A6000 GPUs. To ensure reliability, we conduct three experiments for each sampling point with seed 0, 1, 2 to obtain average Acc. We fix the number of unseen-class examples by setting r = 0.2. Cn is set to 1, 2, 3, 4, 5. We set the imbalance factor (Cib) to 0.01, 0.02, 0.05, 0.10, and 0.20. |