Simultaneous identification of models and parameters of scientific simulators
Authors: Cornelius Schröder, Jakob H. Macke
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate SBMI on a simple time series model and on two scientific models from neuroscience, and show that it can discover multiple data-consistent model configurations, and that it reveals non-identifiable model components and parameters. |
| Researcher Affiliation | Academia | 1Machine Learning in Science, University of Tübingen and Tübingen AI Center, Germany 2Max Planck Institute for Intelligent Systems, Department Empirical Inference, Tübingen, Germany. |
| Pseudocode | Yes | Pseudocode for the sampling procedure and example updating rules can be found in Appendix A3. and See Algorithm 2 for pseudocode. |
| Open Source Code | Yes | Code is available at https://github.com/mackelab/simulation_based_model_inference. |
| Open Datasets | Yes | To apply SBMI to experimental voltage recordings we took ten voltage traces from the Allen Cell Types database (Allen Institute for Brain Science, 2016) previously used in (Gonçalves et al., 2020). and The used data (Roitman & Shadlen, 2002) was collected from two monkeys performing a random dot motion discrimination task. |
| Dataset Splits | Yes | We generated a dataset of 500k prior samples, of which 10% were used as validation data. and From the remaining 180k datapoints we hold back 1k test datapoints and divided the other part into 10% validation and 90% training data. |
| Hardware Specification | Yes | All models were trained on an Nvidia RTX 2080ti GPU accessed via a slurm cluster. |
| Software Dependencies | Yes | All networks were implemented in pytorch (Paszke et al., 2019). Additionally, we used the following software and toolboxes in this work: sbi (Tejero-Cantero et al., 2020) for the implementation of SBMI, Network X (Hagberg et al., 2008) for the construction of prior graphs, Sym Py (Meurer et al., 2017) for symbolic calculations, py DDM (Shinn et al., 2020) as the backend for the DDM experiments. To manage the configuration settings we used Hydra (Yadan, 2019) and the Optuna Sweeper (Akiba et al., 2019) plugin for a coarse hyperparameter search in the DDM setting. |
| Experiment Setup | Yes | The used hyperparameters can be found in Table S5 and S8. and The specifics for the different settings can be found in Table S5, S6, S8, and S13. and For the additive model we used a batch size of 3000 samples, for the DDM a batchsize of 2000 samples, and for the Hodgkin-Huxley model of 4000 samples. |