Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Controlling Multiple Errors Simultaneously with a PAC-Bayes Bound
Authors: Reuben Adams, John Shawe-Taylor, Benjamin Guedj
NeurIPS 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Section 6 outlines positive empirical results from using our bound as a training objective for neural networks and Section 7 gives perspectives for follow-up work.. |
| Researcher Affiliation | Academia | Reuben Adams Department of Computer Science University College London EMAIL; John Shawe-Taylor Department of Computer Science University College London EMAIL; Benjamin Guedj Department of Computer Science, University College London and Inria EMAIL |
| Pseudocode | Yes | Algorithm 1: Calculating a posterior with minimal bound on the total risk. |
| Open Source Code | Yes | Code available here: https://github.com/reubenadams/PAC-Bayes-Control |
| Open Datasets | Yes | We use binarised versions of MNIST, and HAM10000 Tschandl [2018]. For MNIST, we use the conventional training set of size 60000 as the prior set, and the conventional test set of size 10000 as the certification set. |
| Dataset Splits | Yes | For HAM10000 we pool the conventional train, validation and test sets together and then split 50-50 to obtain prior and certification sets each of size 5860. |
| Hardware Specification | No | The paper does not provide specific hardware details for running the experiments. The NeurIPS checklist explicitly states: 'The compute resources required are not stated as they are negligible.' |
| Software Dependencies | No | The paper mentions using MLPs, SGD, and cross-entropy loss, implying the use of deep learning frameworks, but it does not specify any software names with version numbers. |
| Experiment Setup | Yes | We take H to be two-layer MLPs with 784, 100 and 2 units in the input, hidden and output layers, respectively. In both cases we use SGD with learning rate 0.01 to minimise the cross-entropy loss, using a portion of the prior set as a validation set. For MNIST we train the MLP for 20 epochs... For HAM10000 we train the MLP for 5 epochs... |