Approximate Bayesian Computation with Domain Expert in the Loop
Authors: Ayush Bharti, Louis Filstroff, Samuel Kaski
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 4. Experiments In this section, we empirically assess the performance of the proposed HITL-ABC method against regression ABC methods under model misspecification in Section 4.1, and in low-simulation regimes in Section 4.2. Lastly, the sensitivity to hyperparameters is analyzed in Section 4.3. The source code is available at https://github.com/ lfilstro/HITL-ABC. |
| Researcher Affiliation | Academia | 1Department of Computer Science, Aalto University, Espoo, Finland 2Department of Computer Science, University of Manchester, Manchester, United Kingdom. |
| Pseudocode | Yes | Algorithm 1 Human-in-the-loop (HITL) ABC |
| Open Source Code | Yes | The source code is available at https://github.com/ lfilstro/HITL-ABC. |
| Open Datasets | No | The paper describes generating data for its experiments (e.g., 'samples from a g-and-k distribution', 'Radio channel data is measured...') but does not provide concrete access information (link, DOI, specific repository, or formal citation with authors/year) for a publicly available or open dataset. |
| Dataset Splits | No | The paper describes simulating data for experiments but does not provide specific details on train/validation/test data splits (percentages, sample counts, or citations to predefined splits) for its experimental setup. |
| Hardware Specification | No | The paper mentions 'computational resources provided by the Aalto Science-IT Project from Computer Science IT' but does not specify exact GPU/CPU models, processor types, or memory amounts used for running experiments. |
| Software Dependencies | No | The paper mentions using 'the abc R package (Csill ery et al., 2012)' but does not specify a version number for this package or any other software dependencies. |
| Experiment Setup | Yes | Our algorithm can be implemented on top of any ABC method. To identify the effect of the novel contribution, we choose the same method used as a baseline method in comparisons, namely the regression adjustment approach of Beaumont et al. (2002) (linear-ABC). The regression-ABC methods are implemented using the abc R package (Csill ery et al., 2012). For all the experiments, we assume bounded uniform priors on the parameters and use a logit transform (Blum & Franc ois, 2010) before adjusting them to ensure adjusted parameters do not fall outside the prior range. The statistics are normalized by an estimate of their mean absolute deviation before computing the distance to account for the difference in magnitudes. The confidence in the feedback is set to π = 0.95, and the stopping criterion is δ = 0.06. Assuming each statistic is equally likely to be included or excluded a priori, we set ρ = 0.5. We assume ϱ( , ) to be the Euclidean norm , as is a typical choice in ABC. Lastly, a run of the algorithm uses the same simulated data at each iteration for computational ease. |