Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Sharpness-diversity tradeoff: improving flat ensembles with SharpBalance
Authors: Haiquan Lu, Xiaotian Liu, Yefan Zhou, Qunli Li, Kurt Keutzer, Michael W. Mahoney, Yujun Yan, Huanrui Yang, Yaoqing Yang
NeurIPS 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically, we conducted comprehensive evaluations in various data sets (CIFAR-10, CIFAR-100, Tiny Image Net) and showed that Sharp Balance not only effectively improves the sharpness-diversity trade-off, but also significantly improves ensemble performance in ID and OOD scenarios. |
| Researcher Affiliation | Academia | 1 Nankai University 2 Dartmouth College 3 University of California San Diego 4 University of California at Berkeley 5 International Computer Science Institute 6 Lawrence Berkeley National Laboratory |
| Pseudocode | No | The paper describes the Sharp Balance method in text and with a system diagram (Figure 6) but does not include a formally labeled 'Pseudocode' or 'Algorithm' block. |
| Open Source Code | Yes | Our code has been made open-source. https://github.com/haiquanlu/Sharp Balance |
| Open Datasets | Yes | We evaluated this trade-off using a variety of image classification datasets, including CIFAR-10, CIFAR-100 [Krizhevsky, 2009], Tiny Image Net [Le and Yang, 2015], and their corrupted versions [Hendrycks and Dietterich, 2019b]. |
| Dataset Splits | Yes | In addition, we use 10% of the training set as the validation set for selecting ρ and k based on the ensemble s performance. |
| Hardware Specification | Yes | All codes are implemented in Py Torch, and the experiments are conducted on 3 Nvidia Quadro RTX 6000 GPUs for training an ensemble of 3 models. |
| Software Dependencies | No | All codes are implemented in Py Torch. No specific version number for PyTorch or other libraries is provided. |
| Experiment Setup | Yes | We use a batch size of 128, a momentum of 0.9, and a weight decay of 0.0005 for model training. |