Partial Optimal Transport Based Out-of-Distribution Detection for Open-Set Semi-Supervised Learning
Authors: Yilong Ren, Chuanwen Feng, Xike Xie, S. Kevin Zhou
IJCAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We extensively evaluate our proposal under various datasets and OSSL configurations, consistently demonstrating the superior performance of our proposal. ...4 Experimental Results Metrics. To evaluate the performance of OSSL methods, we employ closed-set classification accuracy to test the performance concerning the known classes. We use AUROC to assess the model s open-set classification ability in distinguishing between inliers and outliers. Baselines. In terms of OSSL baselines, we evaluate our approach in comparison to various existing methods, including Fix Match, MTCF, T2T, Open Match, and IOMatch. |
| Researcher Affiliation | Academia | Yilong Ren1,4 , Chuanwen Feng2,4 , Xike Xie3,4 , S. Kevin Zhou3,4,5 1School of Artificial Intelligence and Data Science, University of Science and Technology of China 2School of Computer Science, University of Science and Technology of China (USTC) 3School of Biomedical Engineering, University of Science and Technology of China (USTC) 4Data Darkness Lab, MIRACLE Center, Suzhou Institute for Advanced Research, USTC 5Key Laboratory of Precision and Intelligent Chemistry, USTC |
| Pseudocode | Yes | Algorithm 1 POT Algorithm 2 Mass Score Function Algorithm 3 OSSL Computing Framework |
| Open Source Code | Yes | Codes are available at https://github.com/ryl0427/Code for POT OSSL. |
| Open Datasets | Yes | We evaluate the performance of POT in comparison to baselines on the widely used benchmark datasets for SSL, namely CIFAR10 and CIFAR100. ...we use Image Net-30 dataset [Hendrycks and Gimpel, 2016], a subset of Image Net |
| Dataset Splits | No | The paper describes how known and unknown classes are defined for CIFAR10, CIFAR100, and ImageNet-30 datasets (e.g., 'For CIFAR10, we divide it into 6 known classes and 4 unknown classes.'), and mentions specific numbers of labeled samples per class (e.g., '25 labeled samples per class'), but it does not provide explicit percentages or counts for training, validation, and testing dataset splits. |
| Hardware Specification | No | The paper mentions that experiments were executed 'on a single GPU' but does not provide specific details such as the GPU model, CPU type, or memory specifications used for the experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies or library versions (e.g., Python, PyTorch, CUDA versions) used for the implementation or experiments. |
| Experiment Setup | Yes | In particular, for CIFAR10, we divide it into 6 known classes and 4 unknown classes. For CIFAR100, we consider two settings: one with 80 known classes and 20 unknown classes, and another with 55 known classes and 45 unknown classes, organized according to superclasses. Note that we use the same set of hyper-parameters for all experiments2. 2{µ = 2, B = 64, Ne = 512, Ni = 1024, ε = 0.05, k = 2, λOOD = 0.01}. B indicates the batch size and µ is the relative size of batch size for unlabeled data. Ne indicates the total number of training epochs and Ni is the number of iterations per epoch. ...We adapt a randomly initialized Wide Res Net-28-2 [Zagoruyko and Komodakis, 2016] with 1.5M parameters, in consistency with existing works. In particular, for CIFAR10, we divide it into 6 known classes and 4 unknown classes. ...We employ the Res Net-18 [He et al., 2016] as the backbone network, the remaining experimental settings are the same as those conducted on the CIFAR dataset. |