Direct Amortized Likelihood Ratio Estimation
Authors: Adam D. Cobb, Brian Matejek, Daniel Elenius, Anirban Roy, Susmit Jha
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We benchmark our new ratio estimator and compare it to ratio estimators in the current literature. We show that our new ratio estimator often outperforms these previous approaches. As a further contribution, we introduce a new derivative estimator for likelihood ratio estimators that enables us to compare likelihood-free Hamiltonian Monte Carlo (HMC) with random-walk Metropolis-Hastings (MH). We show that HMC is equally competitive, which has not been previously shown. Finally, we include a novel real-world application of SBI using our neural ratio estimator to design a quadcopter. |
| Researcher Affiliation | Industry | Neuro-Symbolic Computing and Intelligence (Nu SCI) Research Group Computer Science Laboratory, SRI International adam.cobb@sri.com |
| Pseudocode | Yes | Algorithm 1: Optimization of dφ(x, , 0) |
| Open Source Code | Yes | Code is available at https://github.com/SRI-CSL/dnre. |
| Open Datasets | Yes | For the next experiment, we work with the two moons dataset, which is a typical low-dimensional benchmark within the SBI literature (Greenberg, Nonnenmacher, and Macke 2019; Lueckmann et al. 2021). ... In this section we perform a quantitative comparison between the three ratio estimators, where we use the SBI Benchmark examples of Lueckmann et al. (2021) (see Appendix B). ... Finally, we include a novel real-world application of SBI using our neural ratio estimator to design a quadcopter. Code is available at https://github.com/SRI-CSL/dnre. |
| Dataset Splits | Yes | We use a training set of size 10,000 and a validation set of size 5,000. ... We train all three neural ratio estimators using a five layer fully connected architecture with 128 hidden units per layer. We use a learning rate of 0.001 and train all models for 2,000 epochs using a batch size of 512 with 3,592 training points and 898 validation points. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., CPU, GPU models, memory, or specific computing cluster names) used for running the experiments. |
| Software Dependencies | No | The baseline approaches are implemented using the LAMPE Python package (Rozet 2022). While a package is mentioned, specific version numbers for LAMPE or Python are not provided. |
| Experiment Setup | Yes | All models were trained over 1000 epochs with neural network architectures of three layers of 64 units. We use a training set of size 10,000 and a validation set of size 5,000. ... All estimators have an architecture of five layers of 64 units, using the Exponential Linear Unit non-linearity between layers. For all approaches we applied the same grid search over both the learning rate and standard deviation of the proposal distribution for the random-walk MH sampling scheme. ... We train all three neural ratio estimators using a five layer fully connected architecture with 128 hidden units per layer. We use a learning rate of 0.001 and train all models for 2,000 epochs using a batch size of 512 with 3,592 training points and 898 validation points. |