Sequential Bayesian Experimental Design with Variable Cost Structure
Authors: Sue Zheng, David Hayden, Jason Pacheco, John W. Fisher III
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate performance in a Gaussian Markov Random Field (MRF) model which allows for exact evaluation of MI. In this setting, BOEDIR with refinement selection based on marginal utility (mu) performs comparably to non-iterative proxy-based BOED that utilizes either the lower (bed-lb) or upper bound (bed-ub) in Eqn. 4 as a proxy. |
| Researcher Affiliation | Academia | Sue Zheng CSAIL MIT szheng@csail.mit.edu David S. Hayden CSAIL MIT dshayden@csail.mit.edu Jason Pacheco University of Arizona pachecoj@cs.arizona.edu John W. Fisher III CSAIL MIT fisher@csail.mit.edu |
| Pseudocode | Yes | Algorithm 1 Sequential Bayesian Experiment Design; Algorithm 2 BOED with Iterative Refinement of Bounds |
| Open Source Code | No | The paper does not provide an explicit statement or link for open-source code availability. |
| Open Datasets | No | The paper describes generating data for experiments (e.g., 'random trees with |Vx| = 50 nodes', 'scenario with K = 3 targets') but does not provide concrete access information (link, DOI, formal citation) for a publicly available or open dataset. |
| Dataset Splits | No | The paper does not explicitly provide details about training, validation, or test dataset splits. |
| Hardware Specification | No | The paper mentions 'single-chain MCMC' and '16 parallelized chains' in the context of computational cost, but does not specify any particular hardware components (e.g., CPU, GPU models, or memory specifications) used for the experiments. |
| Software Dependencies | No | The paper does not list specific software dependencies with version numbers. |
| Experiment Setup | Yes | We consider posterior costs cp that are 100, 10, 1, and 0.1 the cost cy of sampling ya while holding cy fixed. We generate random trees with |Vx| = 50 nodes, each latent node having two randomly generated candidate projection operators. This results in |Vy| = 100 total candidate experiments to choose from. Following each annotation, we incorporate the new annotation data and draw 3000 posterior samples. We calculate the discrete entropy of this empirical distribution to quantify the ambiguity and repeat for 50 random trials. |