Generalizing Consistent Multi-Class Classification with Rejection to be Compatible with Arbitrary Losses
Authors: Yuzhou Cao, Tianchi Cai, Lei Feng, Lihong Gu, Jinjie GU, Bo An, Gang Niu, Masashi Sugiyama
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 6 Experiments In this section, we provide the experiment results of Cw R with deep models, which are evaluated by the zero-one-c loss following the common practice [44, 8]. We also show the misclassification rate of the accepted data and the ratio of the rejected data. |
| Researcher Affiliation | Collaboration | 1School of Computer Science and Engineering, Nanyang Technological University, Singapore 2Ant Group, China 3College of Computer Science, Chongqing University, China 4RIKEN Center for Advanced Intelligence Project, Japan 5The University of Tokyo, Japan |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [Yes] See the supplemental material. |
| Open Datasets | Yes | In the experiments, we evaluate the proposed methods and baselines on three widely-used benchmarks Fashion-MNIST [60], SVHN [43], CIFAR-10 [31] with cost c selected from {0.05, 0.06, 0.07, 0.08, 0.09, 0.10} for Fashion-MNIST and {0.05, 0.10, 0.15, 0.20, 0.25, 0.30} the other two. |
| Dataset Splits | No | The main text mentions using 'original datasets' and 'data augmentation' but does not provide explicit percentages or counts for train/validation/test splits. It refers to Appendix F for 'Details of the setup', where 'official splits' are mentioned, but no concrete numerical details are given in the main body of the paper for validation or other splits. |
| Hardware Specification | Yes | We implemented all the methods by Pytorch [45], and conducted all the experiments on NVIDIA Ge Force 3090 GPUs. |
| Software Dependencies | No | The paper states, 'We implemented all the methods by Pytorch [45],' but does not specify the version number of Pytorch or any other software dependencies. |
| Experiment Setup | Yes | Details of the setup and the experiments for instance-dependent cost can be found in Appendix F and G, respectively. Appendix F.2 states: 'For each model, we train for 100 epochs with a batch size of 128. We use Adam optimizer [28] with an initial learning rate of 1e-3 and cosine learning rate decay scheduler.' |