On Non-Random Missing Labels in Semi-Supervised Learning
Authors: Xinting Hu, Yulei Niu, Chunyan Miao, Xian-Sheng Hua, Hanwang Zhang
ICLR 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We design an experiment on CIFAR-10 to further illustrate how existing state-of-the-art SSL methods, e.g., Fix Match (Sohn et al., 2020), fail in the MNAR scenario. As shown in Figure 1(a), the overall training data is uniformly distributed over classes, but the labeled data is long-tailed distributed, which simulates the imbalanced class popularity. Trained on such MNAR training data, Fix Match even magnifies the bias towards the popular classes and ignores the rare classes (Figure 1(b)). |
| Researcher Affiliation | Collaboration | Xinting Hu1 Yulei Niu1 Chunyan Miao1 Xian-Sheng Hua2 Hanwang Zhang1 1Nanyang Technological University, 2Damo Academy, Alibaba Group |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide an explicit statement about releasing its own source code or a link to a code repository. |
| Open Datasets | Yes | We evaluate our method on four image classification benchmark datasets: CIFAR-10, CIFAR-100 (Krizhevsky, 2012), STL-10 (Coates et al., 2011) and mini-Image Net (Vinyals et al., 2016). |
| Dataset Splits | Yes | CIFAR-10(-100) is composed of 60,000 images of size 32 32 from 10 (100) classes and each class has 5,000 (500) training images and 1,000 (100) samples for evaluation. STL-10 dataset has 5,000 labeled images and 100,000 unlabeled images of size 64 64. mini-Image Net is a subset of Image Net (Deng et al., 2009). It contains 100 classes where each class has 600 images of size 84 84. Follows previous SSL works (Hu et al., 2021; Iscen et al., 2019), we select 500 images from each class for training and 100 images per class for testing. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory, or cloud instance types) used for running its experiments. |
| Software Dependencies | No | The paper mentions implementing methods based on 'public codes' and 'common network hyper-parameters' but does not specify particular software dependencies with version numbers (e.g., PyTorch version, Python version, specific library versions). |
| Experiment Setup | Yes | Following previous works (Berthelot et al., 2019b; Sohn et al., 2020; Hu et al., 2021), we used Wide Res Net (WRN)-28-2 for CIFAR-10, WRN-28-8 for CIFAR-100, WRN-37-2 for STL-10 and Res Net-18 for mini-Imagenet. Since our methods are implemented as a plug-in module to Fix Match, common network hyper-parameters, e.g., learning rates, batch-sizes, are the same as their original settings (Sohn et al., 2020). For each dataset, our model and Fix Match are trained 217 iterations in MNAR and 220 steps in ordinary SSL cases (γ = 1). |