Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Convex Calibrated Surrogates for Hierarchical Classification
Authors: Harish Ramaswamy, Ambuj Tewari, Shivani Agarwal
ICML 2015 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments on a number of benchmark datasets show that the resulting algorithm, which we term Ov A-Cascade, gives improved performance over other state-of-the-art hierarchical classification algorithms. |
| Researcher Affiliation | Academia | Harish G. Ramaswamy HARISH EMAIL Indian Institute of Science, Bangalore, INDIA Ambuj Tewari EMAIL University of Michigan, Ann Arbor, USA Shivani Agarwal EMAIL Indian Institute of Science, Bangalore, INDIA |
| Pseudocode | Yes | Algorithm 1 OVA-Cascade Training |
| Open Source Code | No | The paper does not provide any specific repository link, explicit code release statement, or mention of code in supplementary materials for the methodology described. |
| Open Datasets | Yes | CLEF (Dimitrovski et al., 2011) Medical X-ray images organized according to a hierarchy. IPC 3 Patents organized according to the International Patent Classification Hierarchy. LSHTC-small, DMOZ-2010 and DMOZ-2012 4 Web-pages, from the LSHTC (Large-Scale Hierarchical Text Classification) challenges 2010-12, organized according to a hierarchy. 3http://www.wipo.int/classifications/ipc/en/support/ 4http://lshtc.iit.demokritos.gr/node/3 |
| Dataset Splits | Yes | Table 1. Dataset Statistics Dataset #Train #Validation #Test #Labels #Leaf-Labels Depth #Features CLEF 9,000 1,000 1,006 97 63 3 89 LSHTC-small 4,463 1,860 1,858 2,388 1,139 5 51,033 IPC 35,000 11,324 28,926 553 451 3 541,869 DMOZ-2010 80,000 13,805 34,905 17,222 12,294 5 381,580 DMOZ-2012 250,000 50,000 83,408 13,347 11,947 5 348,548 |
| Hardware Specification | No | The paper mentions running algorithms 'on a 4-core CPU' but does not provide specific details such as the CPU model, GPU, memory, or other detailed computer specifications. |
| Software Dependencies | No | The paper states 'We use LIBLINEAR (Fan et al., 2008) for the SVM-train subroutine' but does not specify the version number of this or any other software dependency. |
| Experiment Setup | No | The paper mentions that 'The regularization parameter C is chosen via a separate validation set. The thresholds τj for j [h] are also chosen via a coarse grid search using the validation set.' but does not provide the specific hyperparameter values for C or τj. |