Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Misspecification-robust Sequential Neural Likelihood for Simulation-based Inference
Authors: Ryan P. Kelly, David J Nott, David Tyler Frazier, David J Warne, Christopher Drovandi
TMLR 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the efficacy of our approach through several illustrative examples, where our method gives more accurate point estimates and uncertainty quantification than SNL. ... In Figure 2, we illustrate the coverage and log-density at θ0 of SNL, RSNL and RNPE across four simulation examples. |
| Researcher Affiliation | Academia | Ryan P. Kelly EMAIL School of Mathematical Sciences Centre for Data Science Queensland University of Technology David J. Nott EMAIL Institute of Operations Research and Analytics National University of Singapore David T. Frazier EMAIL Department of Econometrics and Business Statistics Monash University David J. Warne EMAIL School of Mathematical Sciences Centre for Data Science Queensland University of Technology Christopher Drovandi EMAIL School of Mathematical Sciences Centre for Data Science Queensland University of Technology |
| Pseudocode | Yes | Algorithm 1 Robust MCMC SNL Input: The observed summaries, S(y); the prior distributions π(θ) and π0(Γ); the number of training rounds, R; the assumed data generating process, P (n) θ ; the number of simulated datasets from P (n) θ generated per round, m; the neural density estimator family, qϕ(S(x) | θ). Output: MCMC samples (θ0, . . . , θm 1) and (Γ0, . . . , Γm 1) from the RSNL posterior. 1: Set D = {}, q0,ϕ(S(y) | θ) = 1 2: for r = 0 to R 1 do 3: Update πr(Γ) when r > 0 4: for i = 0 to m 1 do 5: Sample θ(r) i , Γ(r) i qr,ϕ(S(y) Γ | θ)π(θ)πr(Γ) using MCMC or directly when r = 0 6: Simulate x(r) i P (n) θ(r) i 7: Compute summaries S(x(r) i ) 8: Add (θ(r) i , S(x(r) i )) into D 9: end for 10: Standardise D and S(y) 11: Train qr+1,ϕ(S(x) | θ) on D 12: end for 13: Sample θ(R) i , Γ(R) i q R,ϕ(S(y) Γ | θ)π(θ)πR(Γ) 14: return θ(R) 0:m 1, Γ(R) 0:m 1 |
| Open Source Code | Yes | The code to reproduce the results has been included as supplementary material. |
| Open Datasets | Yes | We consider here the animal movement model by Marchand et al. (2017) to simulate the dispersal of Fowler s toads (Anaxyrus fowleri). ... The simple likelihood complex posterior (SLCP) model devised in Papamakarios et al. (2019) is a popular example in the SBI literature. |
| Dataset Splits | Yes | To calculate the empirical coverage, we generate C = 200 observed summaries S(yi) S(P (n) 0 ) at θ0,i, where θ0,i represents the true data generating parameter and i = 1, . . . , C. ... Flow training is stopped when either the validation loss, calculated on 10% of the samples, has not improved over 20 epochs or when the limit of 500 epochs is reached. |
| Hardware Specification | Yes | All simulations and inference were executed on a high-performance computer, each running using a single Intel Xeon core with 8GB of RAM. |
| Software Dependencies | Yes | The robust sequential neural likelihood (RSNL) inference algorithm was implemented using the JAX (Bradbury et al., 2018) and Num Pyro (Phan et al., 2019) libraries due to the speed of these libraries for MCMC sampling. Plotting was done using the matplotlib library (Hunter, 2007), and MCMC diagnostics and visualisations were done using the Arvi Z library (Kumar et al., 2019). The SIR model is implemented in JAX using the diffrax library (Kidger, 2021). RNPE results were obtained from implementing the example using the publicly available code at https://github.com/danielward27/rnpe. RBSL results were obtained using the ELFI software package implementation (Lintusaari et al., 2018; Kelly, 2022). We utilise a conditional neural spline flow (Durkan et al., 2019) for qϕ(S(x) | θ), as implemented in the flowjax package (Ward, 2023). ... Python package version 0.3.13, URL https://github.com/google/jax. ... Python package version 7.0.0, [Online; accessed 17-January-2023], URL https://github.com/danielward27/flowjax. |
| Experiment Setup | Yes | We use 10 bins over the interval [-5, 5] for the rational quadratic spline transformer. The conditioner consists of five coupling layers, each using a multilayer perceptron of two layers with 50 hidden units. The flow is trained using the Adam optimiser (Kingma & Ba, 2015) with a learning rate of 5 10 4 and a batch size of 256. Flow training is stopped when either the validation loss, calculated on 10% of the samples, has not improved over 20 epochs or when the limit of 500 epochs is reached. |