An Unbiased Risk Estimator for Learning with Augmented Classes
Authors: Yu-Jie Zhang, Peng Zhao, Lanjihong Ma, Zhi-Hua Zhou
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments confirm the effectiveness of our proposed approach. ... We examine three aspects of the proposed EULAC approach: (Q1) performance of classifying known classes and identifying augmented classes; (Q2) accuracy of estimating mixture prior θ and its influence on EULAC; (Q3) capability of handling the complex changing environments (augmented class appears and prior of known classes shifts simultaneously). We answer the questions in following three subsections. In all experiments, classifiers are trained with labeled and unlabeled data, and are evaluated with an additional testing dataset which is never observed in training. |
| Researcher Affiliation | Academia | Yu-Jie Zhang, Peng Zhao, Lanjihong Ma, Zhi-Hua Zhou National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210023, China {zhangyj, zhaop, maljh, zhouzh}@lamda.nju.edu.cn |
| Pseudocode | No | The paper describes the algorithms and their implementation (Kernel-based hypothesis space, Deep model) but does not provide formal pseudocode blocks or algorithms labeled as such. |
| Open Source Code | No | The paper does not provide an explicit statement about open-sourcing the code or a link to a code repository for the described methodology. |
| Open Datasets | Yes | We adopt 9 datasets, where half of the total classes are randomly selected as augmented classes for 10 times. In each dataset, the labeled, unlabeled and testing data contain 500, 1000 and 1000 instance respectively. ... The experiments are conducted on mnist, SVHN and Cifar-10 datasets, where six of all ten classes are randomly selected as known while the rest four are treated as augmented. |
| Dataset Splits | Yes | It is noteworthy to mention that our approach can now perform the standard cross validation procedure to select parameters, while most geometric-based approaches cannot due to the unavailability of the testing distribution, and their parameters setting heavily relies on the experience. |
| Hardware Specification | No | The paper does not specify any hardware used for running the experiments (e.g., GPU/CPU models, memory, or specific computing platforms). |
| Software Dependencies | No | The paper mentions RKHS-based and DNN-based implementations and the use of specific loss functions (e.g., sigmoid loss, logistic loss, square loss), but it does not specify software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | No | The paper mentions aspects of the experimental setup like the sampling procedure (e.g., 'randomly selected as augmented classes for 10 times', 'instance sampling procedure repeats 10 times'), the use of specific loss functions, and how mixture proportions are estimated. However, it does not provide concrete hyperparameter values (e.g., learning rate, batch size, number of epochs) or specific system-level training settings. |