Towards Realistic Model Selection for Semi-supervised Learning
Authors: Muyang Li, Xiaobo Xia, Runze Wu, Fengming Huang, Jun Yu, Bo Han, Tongliang Liu
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In addition, comprehensive experiments showcase that SLAM can achieve significant improvements compared to its counterparts, verifying its efficacy from both theoretical and empirical standpoints. 5. Experiment Datasets. We evaluate our methods on a series of commonly used benchmark datasets in Semi-supervised Learning (SSL): CIFAR-10 and CIFAR-100 (Krizhevsky et al., 2009). 5.2. Results To comprehensively evaluate the capability of SLAM, we conduct several realistic model selection tasks under SSL settings, including early-stopping, hyper-parameter selection, and model selection against train/val splitting. |
| Researcher Affiliation | Collaboration | 1Sydney AI Center, The University of Sydney 2FUXI AI Lab, Net Ease 3University of Science and Technology of China 4Hong Kong Baptist University. |
| Pseudocode | Yes | Algorithm 1 Local-consistency re-weighting. |
| Open Source Code | No | The paper does not contain an explicit statement or link indicating the release of open-source code for the proposed methodology. |
| Open Datasets | Yes | Datasets. We evaluate our methods on a series of commonly used benchmark datasets in Semi-supervised Learning (SSL): CIFAR-10 and CIFAR-100 (Krizhevsky et al., 2009). |
| Dataset Splits | Yes | To create different level of available supervision signals, following default setting (Sohn et al., 2020; Wang et al., 2022b), for CIFAR-10, we randomly sample {4,25,400} labeled data per-class, for CIFAR-100, we randomly sample {4,25,100} labeled data per-class. More specifically, we consider cases where there are more than 10 labeled data per class. We test two validation split ratios, 10% and 20%. |
| Hardware Specification | No | This research was undertaken with the assistance of resources from the National Computational Infrastructure (NCI Australia), an NCRIS enabled capability supported by the Australian Government. This work was supported by resources provided by the Pawsey Supercomputing Research Centre s Setonix Supercomputer, with funding from the Australian Government and the Government of Western Australia. The authors acknowledge the technical assistance provided by the Sydney Informatics Hub, a Core Research Facility of the University of Sydney. |
| Software Dependencies | No | The paper discusses model architectures like "Wide Res Net-28-2" but does not specify any software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow, CUDA versions). |
| Experiment Setup | Yes | Implementation details. We follow the commonly used implementation details in SSL (Berthelot et al., 2019b; Sohn et al., 2020; Zhang et al., 2021a), where we use Wide Res Net28-2 (Zagoruyko & Komodakis, 2016) for CIFAR-10... All models are trained for 220 iterations. For instance, Mix Match s critical parameters include the weights assigned to unlabeled loss (Berthelot et al., 2019b), we define a search range for these weights set at {1, 25, 50, 100}. Re Mix Match builds upon Mix Match by introducing additional parameters, such as the weight controlling the rotation loss (Berthelot et al., 2019a), with its search range defined as {0.1, 0.25, 0.5, 1}. Similarly, for Fix Match, significant parameters include the confidence threshold (Sohn et al., 2020), with its search range specified as {0.5, 0.8, 0.9, 0.95}. |