Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Amortized Bayesian Decision Making for simulation-based models
Authors: Mila Gorecki, Jakob H. Macke, Michael Deistler
TMLR 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We apply our method to several benchmark problems and demonstrate that it induces similar cost as the true posterior distribution. We then apply the method to infer optimal actions in a real-world simulator in the medical neurosciences, the Bayesian Virtual Epileptic Patient, and demonstrate that it allows to infer actions associated with low cost after few simulations. |
| Researcher Affiliation | Academia | Mila Gorecki EMAIL Social Foundations of Computation, Max Planck Institute for Intelligent Systems, Tübingen Tübingen AI Center Jakob H. Macke EMAIL Machine Learning in Science, Excellence Cluster Machine Learning, University of Tübingen Empirical Inference, Max Planck Institute for Intelligent Systems, Tübingen Tübingen AI Center Michael Deistler EMAIL Machine Learning in Science, Excellence Cluster Machine Learning, University of Tübingen Tübingen AI Center |
| Pseudocode | Yes | Algorithm 1: Bayesian Amortized decision Making (BAM) |
| Open Source Code | Yes | Code to reproduce all experiments, including the full git commit history, is available at https://github. com/mackelab/amortized-decision-making. |
| Open Datasets | Yes | We used the toy example introduced above and three previously published simulators with ground truth posteriors (Lueckmann et al., 2021). The framework, termed the Bayesian Virtual Epileptic Patient (BVEP), allows to simulate neural activity in a connected network of brain regions to model epilepsy spread (Hashemi et al., 2020; Jirsa et al., 2017). Previous work has demonstrated that NPE is close to an approximate ground truth (obtained with Hamiltonian Monte-Carlo) in this model (Hashemi et al., 2023). |
| Dataset Splits | Yes | In all cases, the training dataset was split 90:10 into training and validation and with a batchsize of 500. |
| Hardware Specification | Yes | Exemplary runtimes for the multi-dimensional Lotka-Volterra task measured on a NVIDIA Ge Force RTX 2080 Ti GPU. |
| Software Dependencies | No | All neural networks and optimization loops are written in pytorch (Paszke et al., 2019). We tracked all experiments with hydra (Yadan, 2019). For NPE, we used the implementation in the sbi toolbox (Tejero-Cantero et al., 2020). |
| Experiment Setup | Yes | Only the learning rate was adjusted to 0.005 for Lotka-Volterra, while it was set to 0.001 for all other tasks. For BAM, we used a feedforward residual network (He et al., 2016) with 3 hidden layers and 50 hidden units each. We use Re LU activation functions and, for cases where we know that the expected cost is going to be positive and bounded by 1, squash the output through a sigmoid. We use the Adam optimizer (Kingma and Ba, 2015). In all cases, the training dataset was split 90:10 into training and validation and with a batchsize of 500. |