Binary Classification with Karmic, Threshold-Quasi-Concave Metrics
Authors: Bowei Yan, Sanmi Koyejo, Kai Zhong, Pradeep Ravikumar
ICML 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | In this paper, we advance this understanding of binary classification for complex performance measures by identifying two key properties: a so-called Karmic property, and a more technical threshold-quasiconcavity property, which we show is milder than existing structural assumptions imposed on performance measures. Under these properties, we show that the Bayes optimal classifier is a threshold function of the conditional probability of positive class. We then leverage this result to come up with a computationally practical plug-in classifier, via a novel threshold estimator, and further, provide a novel statistical analysis of classification error with respect to complex performance measures. |
| Researcher Affiliation | Academia | 1University of Texas at Austin, Austin, Texas, USA 2University of Illinois at Urbana-Champaign, Champaign, Illinois, USA 3Carnegie Mellon University, Pittsburgh, Pennsylvania, USA. |
| Pseudocode | Yes | Algorithm 1 Two-step Plug-in Classifier for General Metrics |
| Open Source Code | No | The paper does not provide concrete access to source code for the methodology described, nor does it state that code is available. |
| Open Datasets | No | The paper does not provide concrete access information for a publicly available or open dataset. It refers to theoretical models such as the 'Gaussian generative model' and 'β-H older densities' for analytical purposes rather than specific datasets for experimentation. |
| Dataset Splits | No | The paper does not provide specific dataset split information as it focuses on theoretical analysis and does not report on empirical experiments with datasets. |
| Hardware Specification | No | The paper does not provide specific hardware details used for running experiments, as the research presented is theoretical and does not report on empirical experiments. |
| Software Dependencies | No | The paper does not provide specific ancillary software details (e.g., library or solver names with version numbers), as the research presented is theoretical and does not report on empirical experiments. |
| Experiment Setup | No | The paper does not contain specific experimental setup details (e.g., hyperparameters, training configurations), as the research presented is theoretical and does not report on empirical experiments. |