Optimizing Non-decomposable Performance Measures: A Tale of Two Classes
Authors: Harikrishna Narasimhan, Purushottam Kar, Prateek Jain
ICML 2015 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, we present an empirical validation of our methods. Our experiments reveal that for a range of performance measures in both classes, our methods can be significantly faster than either plug-in or SVMPerf-style methods, as well as give higher or comparable accuracies. |
| Researcher Affiliation | Collaboration | Harikrishna Narasimhan HARIKRISHNA@CSA.IISC.ERNET.IN Indian Institute of Science, Bangalore, INDIA Purushottam Kar T-PURKAR@MICROSOFT.COM Microsoft Research, INDIA Prateek Jain PRAJAIN@MICROSOFT.COM Microsoft Research, INDIA |
| Pseudocode | Yes | Algorithm 1 SPADE: Stochastic Prim Al-Dual m Ethod |
| Open Source Code | No | The paper does not contain an explicit statement about the availability of open-source code for the described methodology, nor does it provide any links to a code repository. |
| Open Datasets | Yes | Datasets: We evaluated our methods on 5 publicly available benchmark datasets: a) PPI, b) KDD Cup 2008, c) IJCNN, d) Covertype, e) MNIST. |
| Dataset Splits | Yes | We used 70% of the dataset for training and the rest for testing. Tunable parameters, including thresholds for the plug-in approaches, were cross-validated on a validation set. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., exact GPU/CPU models, processor types, or memory amounts) used for running its experiments. |
| Software Dependencies | No | The paper states 'All methods were implemented in C.' and mentions using an 'LBFGS solver' and 'standard implementations of the SVMPerf algorithm', but does not provide specific version numbers for these or other software dependencies. |
| Experiment Setup | Yes | Tunable parameters, including thresholds for the plug-in approaches, were cross-validated on a validation set. ... We used hinge-loss based reward functions for our methods. ... SPADE was allowed to run for 25 passes over the data and STAMP was allowed 25 passes with an initial epoch length of 100 which was doubled after every iteration. |