Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
OCCAM: Towards Cost-Efficient and Accuracy-Aware Classification Inference
Authors: Dujian Ding, Bicheng Xu, Laks Lakshmanan
ICLR 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | On a variety of real-world datasets, OCCAM achieves 40% cost reduction with little to no accuracy drop. |
| Researcher Affiliation | Academia | Dujian Ding, Bicheng Xu, Laks V. S. Lakshmanan University of British Columbia EMAIL EMAIL |
| Pseudocode | Yes | Algorithm 1: OCCAM Algorithm. Input: test query batch X; ML classifiers f1, f2, , f M and costs c1, c2, , c M; query samples S1, S2, ..., Sk; user cost budget B. Output: optimal model portfolio ยต : X [M]. |
| Open Source Code | Yes | Codes are available in https://github.com/Dujian Ding/OCCAM.git. |
| Open Datasets | Yes | We consider 4 widely studied datasets for image classification: CIFAR-10 (10 classes) (Krizhevsky et al., 2009), CIFAR-100 (100 classes) (Krizhevsky et al., 2009), Tiny Image Net (200 classes) (CS231n), and Image Net-1K (1000 classes) (Russakovsky et al., 2015). |
| Dataset Splits | Yes | We randomly sample 20, 000 images from the training set as our validation set, and we use the remaining 30, 000 images to train our models. (from CIFAR-10 description in C.1). |
| Hardware Specification | Yes | All experiments are conducted with one NVIDIA V100 GPU of 32GB GPU RAM. |
| Software Dependencies | No | The paper mentions the Adam optimizer and Hi GHS ILP solver, but does not provide specific version numbers for these software components or any other libraries like PyTorch. |
| Experiment Setup | Yes | For all seven models, we use the Adam optimizer (Kingma & Ba, 2015) with ฮฒ1 = 0.9 and ฮฒ2 = 0.999, constant learning rate 0.00001, and a batch size of 500 for training. Models are trained till convergence. |