Learning via Surrogate PAC-Bayes
Authors: Antoine Picard, Roman Moscoviz, Benjamin Guedj
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | illustrate our approach with numerical experiments inspired by an industrial biochemical problem. |
| Researcher Affiliation | Collaboration | Antoine Picard-Weibel Inria & SUEZ , France antoine.picard.ext@suez.com Roman Moscoviz SUEZ , France roman.moscoviz@suez.com Benjamin Guedj Inria and University College London , France and United Kingdom benjamin.guedj@inria.fr |
| Pseudocode | Yes | Algorithm 1 Surrogate PAC-Bayes Learning framework (Su PAC) Require: PB, π0 Π, πp P, R M(H) π π0 while not converged do f F(π, R) π Solve(πp, π, f) end while |
| Open Source Code | Yes | Following Theorem 2, we propose an algorithm, Su PAC-CE (https://github.com/ APicard Weibel/surpbayes), designed to efficiently find the minimiser of Catoni s bound on Exponential families. |
| Open Datasets | Yes | We compared Su PAC-CE to standard GD on a synthetic dataset from Picard-Weibel et al. [2024], using the same family of distributions and risk function. |
| Dataset Splits | No | The paper mentions training procedures and evaluating performance on test tasks, but it does not explicitly describe a validation dataset split (e.g., in percentages or specific sample counts) or cross-validation setup for model selection during training. |
| Hardware Specification | Yes | Computations were performed using Azure Machine Learning compute clusters with 32 cores and Intel Xeon Platinum 8272CL processors. Computations were performed using Azure Machine Learning compute clusters with 16 cores and Intel Xeon Platinum 8272CL processors. |
| Software Dependencies | No | The paper mentions using the 'Faiss library' but does not specify its version number or the versions of other core software components like Python, specific machine learning frameworks, or compilers. |
| Experiment Setup | Yes | For Su PAC-CE, 160 risk queries where performed for the initial step, and 32 for all further step. A maximal budget of 9600 empirical risk queries was fixed; hyperparameters for the GD were selected after evaluating a grid on the first 1600 queries. The PAC-Bayes temperature was set to 0.002. For Su PAC-CE, the regularisation hyperparameters were set to klmax = 1 and αmax = 0.5, while the number of samples generated to evaluate the weights was set to 40 000. Hyperparameters for GD were selected after assessing the grid per_step {80, 160}, step_size {0.025, 0.05, 0.07} on a preliminary 1600 score queries budget, with 20 repeats. |