Sampling-Based Accuracy Testing of Posterior Estimators for General Inference
Authors: Pablo Lemos, Adam Coogan, Yashar Hezaveh, Laurence Perreault-Levasseur
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We apply our algorithm, described in Algorithm 2 to three different experiments. |
| Researcher Affiliation | Academia | 1Mila Quebec AI Institute, Montreal, Quebec, Canada 2Universit e de Montr eal, Montreal, Quebec, Canada 3CIELA Institute, Montreal, Quebec, Canada 4Flatiron Institute Center for Computational Mathematics, 162 5th Ave, 3rd floor, New York, NY 10010, USA. Correspondence to: Pablo Lemos <pablo.lemos@umontreal.ca>. |
| Pseudocode | Yes | Algorithm 1 Calculation of ECP(ˆp, α, H) using highest posterior density regions, from a set of simulations {θi, xi}, i [1, Nsims] Algorithm 2 Calculation of ECP(ˆp, α, Dθr) using the TARP method, using a set of simulations {θi, xi}, i {1, . . . , Nsims}, parameter distance metric d : U U R 0 and reference point sampling distribution p( |x). |
| Open Source Code | Yes | Our code is available at https://github.com/Ciela-Institute/tarp. |
| Open Datasets | Yes | The simulator in this scenario samples the source galaxy s light θ from a multivariate-normal distribution that we fit to a dataset of galaxy images (Stone & Courteau, 2019; Stone et al., 2021). |
| Dataset Splits | No | The paper describes generating simulations and using them for validation (e.g., 'We first generate simulations', 'For each of these cases, we want to compare...'). However, it does not specify explicit train/validation/test splits with percentages or sample counts for any of the datasets used, nor does it refer to predefined splits. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running the experiments (e.g., CPU/GPU models, memory specifications). |
| Software Dependencies | No | The paper does not list specific software dependencies with version numbers (e.g., 'Python 3.8, PyTorch 1.9'). |
| Experiment Setup | Yes | We find 300 steps are sufficient to ensure convergence. |