Learning Robust Statistics for Simulation-based Inference under Model Misspecification
Authors: Daolang Huang, Ayush Bharti, Amauri Souza, Luigi Acerbi, Samuel Kaski
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the superior performance of our method on high-dimensional time-series models that are artificially misspecified. We also apply our method to real data from the field of radio propagation where the model is known to be misspecified. We show empirically that the method yields robust inference in misspecified scenarios, whilst still being accurate when the model is well-specified. We apply our method of learning robust statistics to two different SBI frameworks NPE [41] and ABC [5] (see Appendix B.6 for results on applying our method to neural likelihood estimator [52]). We also compare the performance of our method against RNPE [81]... |
| Researcher Affiliation | Academia | Daolang Huang Aalto University daolang.huang@aalto.fi Ayush Bharti Aalto University ayush.bharti@aalto.fi Amauri H. Souza Aalto University Federal Institute of Ceara amauri.souza@aalto.fi Luigi Acerbi University of Helsinki luigi.acerbi@helsinki.fi Samuel Kaski Aalto University University of Manchester samuel.kaski@aalto.fi |
| Pseudocode | No | The paper describes methods like rejection-ABC and estimation of squared-MMD in paragraph form, but it does not include any explicitly labeled 'Pseudocode' or 'Algorithm' blocks, nor does it present structured steps formatted like code. |
| Open Source Code | Yes | The code to reproduce our experiments is available at https://github.com/huangdaolang/robust-sbi. |
| Open Datasets | No | We also apply our method to real data from the field of radio propagation where the model is known to be misspecified. We attempt to fit this model to a real dataset from a measurement campaign [44]. (The paper acknowledges that the data was provided by specific individuals, implying it is not generally publicly available with a direct link/DOI/citation for access. For synthetic data, it is generated, not from a public dataset.) |
| Dataset Splits | No | We take m = 1000 samples for the training data and n = 100 realizations of both the observed and simulated data for each θ. Choosing λ can be cast as a hyperparameter selection problem, for which we can leverage additional data (if available) as a validation dataset, or use post-hoc qualitative analysis of the posterior predictive distribution. (The paper states that validation data can be used but does not specify that it was used with a specific split for their experiments.) |
| Hardware Specification | Yes | The results, presented in Table 2, are calculated on an Apple M1 Pro CPU. |
| Software Dependencies | No | We implement our NPE-RS models based on publicly available implementations from https://github.com/mackelab/sbi. We use the NPE-C model [41] with Masked Autoregressive Flow (MAF) [60] as the backbone inference network, and adopt the default configuration with 50 hidden units and 5 transforms for MAF. The batch size is set to 50, and we maintain a fixed learning rate of 5e-4. The implementation for RNPE is sourced directly from the original repository at https://github.com/danielward27/rnpe. (No specific version numbers for software dependencies like PyTorch, Python, or the mentioned frameworks are provided.) |
| Experiment Setup | Yes | We use the NPE-C model [41] with Masked Autoregressive Flow (MAF) [60] as the backbone inference network, and adopt the default configuration with 50 hidden units and 5 transforms for MAF. The batch size is set to 50, and we maintain a fixed learning rate of 5e-4. For the Ricker model, the summary network ηψ is composed of 1D convolutional layers, whereas for the OUP, ηψ is a combination of bidirectional long short-term memory (LSTM) recurrent modules and 1D convolutional layers. The dimension of the statistic space is set to four for both the models. |