Cost-Sensitive Self-Training for Optimizing Non-Decomposable Metrics
Authors: Harsh Rangwani, shrinivas ramasubramanian, Sho Takemori, Kato Takashi, Yuhei Umeda, Venkatesh Babu R
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our results demonstrate that CSST achieves an improvement over the state-of-the-art in majority of the cases across datasets and objectives. We demonstrate that the proposed CSST framework shows significant gains in performance on both vision and NLP tasks on imbalanced datasets, with an imbalance ratio defined on the training set as ρ = maxi P (y=i) mini P (y=i) . |
| Researcher Affiliation | Collaboration | 1Video Analytics Lab, Indian Institute of Science, Bengaluru, India 2Fujitsu Limited, Kanagawa, Japan |
| Pseudocode | Yes | See Alg. 1 in Appendix |
| Open Source Code | Yes | Code: https://github.com/val-iisc/Cost Sensitive Self Training |
| Open Datasets | Yes | For CIFAR-10 [11], IMDb [16] and DBpedia-14 [14], we use ρ = 100 and ρ = 10 for CIFAR-100 [11] and Image Net-100 [27] 2 datasets. |
| Dataset Splits | Yes | We divide the balanced held-out set for each dataset equally into validation and test sets. |
| Hardware Specification | No | The paper states 'See Section G in Appendix' for details on hardware specification, but these details are not provided in the main body of the paper. |
| Software Dependencies | No | The paper mentions using Wide Res Nets(WRN), Res Net-50, and Distil BERT, but does not provide specific version numbers for software dependencies or libraries. |
| Experiment Setup | No | The paper states 'A detailed list of hyper-parameters and additional experiments can be found in the Appendix Tab. 4 and Sec. O respectively', but these details are not explicitly provided in the main text. |