Towards Reliable Simulation-Based Inference with Balanced Neural Ratio Estimation
Authors: Arnaud Delaunoy, Joeri Hermans, François Rozet, Antoine Wehenkel, Gilles Louppe
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate BNRE on a wide variety of tasks and show that it produces conservative posterior surrogates on all tested benchmarks and simulation budgets. We start by providing an extensive validation of BNRE on a broad range of benchmarks demonstrating that the proposed method alleviates the problem. |
| Researcher Affiliation | Academia | Arnaud Delaunoy University of Liège a.delaunoy@uliege.be Joeri Hermans* Unaffiliated joeri@peinser.com François Rozet University of Liège francois.rozet@uliege.be Antoine Wehenkel University of Liège antoine.wehenkel@uliege.be Gilles Louppe University of Liège g.louppe@uliege.be |
| Pseudocode | Yes | Algorithm 1 Training algorithm for Balanced Neural Ratio Estimation (BNRE). Inputs: Implicit generative model p(x | ϑ) (simulator) and prior p(ϑ) Outputs: Approximate classifier ˆdψ(ϑ, x) parameterized by ψ hyper-parameters: Balancing condition strength λ (default = 100) and batch-size n Sample data from the joint {ϑi, xi p(ϑ, x), yi = 1}n/2 i=1 Sample data from the marginals {ϑi, xi p(ϑ)p(x), yi = 0}n i=n/2+1 L[ ˆdψ] = 1 n Pn i=1 yi log ˆdψ(ϑi, xi) + (1 yi) log(1 ˆdψ(ϑi, xi)) B[ ˆdψ] = 2 n Pn/2 i=1 ˆdψ(ϑi, xi) + 2 n Pn i=n/2+1 ˆdψ(ϑi, xi) ψ = minimizer_step(params=ψ, loss=L[ ˆdψ] + λ(B[ ˆdψ] 1)2) until convergence return ˆdψ(ϑ, x). |
| Open Source Code | Yes | Code is available at https://github.com/montefiore-ai/balanced-nre. |
| Open Datasets | Yes | We evaluate the expected coverage of posterior estimators produced by both NRE and BNRE on various problems. Those benchmarks cover a diverse set of problems from particle physics (Weinberg), epidemiology (Spatial SIR), queueing theory (M/G/1), population dynamics (Lotka Volterra, and astronomy (Gravitational Waves). They are representative of real scientific applications of simulation-based inference. A more detailed description of the benchmarks can be found in Appendix C. |
| Dataset Splits | No | No explicit mention of a separate validation dataset split. The paper mentions "simulation budgets" for training and evaluates on "10000 unseen samples". |
| Hardware Specification | No | No specific hardware details like GPU/CPU models or memory specifications are provided. The paper only mentions 'Computational resources have been provided by the Consortium des Équipements de Calcul Intensif (CÉCI)'. |
| Software Dependencies | No | The paper describes neural network architectures and training hyperparameters (Appendix D) but does not list specific software dependencies with version numbers (e.g., PyTorch 1.x, Python 3.x). |
| Experiment Setup | Yes | The architectures and hyper-parameters used for each problem are defined in Appendix D. (e.g., learning rate 10^-3, batch size 2^10, 500 epochs, Adam optimizer). |