Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Weighted ROC Curve in Cost Space: Extending AUC to Cost-Sensitive Learning
Authors: HuiYang Shao, Qianqian Xu, Zhiyong Yang, Peisong Wen, Gao Peifeng, Qingming Huang
NeurIPS 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct extensive experiments on multiple imbalanced cost-sensitive classification tasks. The experimental results speak to the effectiveness of our proposed methods. In this section, we conduct a series of experiments for WAUC cost-sensitive learning on common long-tail benchmark datasets. |
| Researcher Affiliation | Academia | 1 Key Lab. of Intelligent Information Processing, Institute of Computing Tech., CAS 2 School of Computer Science and Tech., University of Chinese Academy of Sciences 3 BDKM, University of Chinese Academy of Sciences |
| Pseudocode | Yes | Algorithm 1 Stochastic Algorithm for WAUC Cost-sensitive Learning |
| Open Source Code | Yes | The source code is available in supplemental materials. |
| Open Datasets | Yes | We use three datasets: Binary CIFAR-10-Long-Tail Dataset [23], Binary CIFAR-100-Long-Tail Dataset [23], and Jane Street Market Prediction [14]. |
| Dataset Splits | Yes | For all datasets, we divide them into the training set, validation set, and test set with a proportion 0.7:0.15:0.15. |
| Hardware Specification | No | No specific hardware details (e.g., CPU/GPU model, memory) used for running experiments were mentioned in the paper. |
| Software Dependencies | No | No specific software dependencies with version numbers (e.g., Python 3.8, PyTorch 1.9) were explicitly provided in the paper. |
| Experiment Setup | Yes | In this subsection, we show the sensitivity of β, T, and bandwidth on test data. Effect of β. Effect of T. Effect of m. |