Dynamic Causal Bayesian Optimization
Authors: Virginia Aglietti, Neil Dhir, Javier González, Theodoros Damoulas
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate the performance of DCBO in a variety of synthetic and real world settings with DAGs given in Fig. 3. We first run the algorithm for a stationary setting where both the graph structure and the SCM do not change over time (STAT.). We then consider a scenario characterised by increased observation noise (NOISY) for the manipulative variables and a settings where observational data are missing at some time steps (MISS.). Still assuming stationarity, we then test the algorithm in a DAG where there are multivariate interventions in Mt (MULTIV.). Finally, we run DCBO for a non-stationary graph where both the SCM and the DAG change over time (NONSTAT.). To conclude, we use DCBO to optimize the unemployment rate of a closed economy (DAG in Fig. 3d, ECON.) and to find the optimal intervention in a system of ordinary differential equation modelling a real predator-prey system (DAG in Fig. 3e, ODE). We provide a discussion on the applicability of DCBO to real-world problems in 7 of the supplement together with all implementation details. Baselines We compare against the algorithms in Fig. 1. Note that, by constructions, ABO and BO intervene on all manipulative variables while DCBO and CBO explore only Mt at every t. In addition, both DCBO and ABO reduce to CBO and BO at the first time step. We assume the availability of an observational dataset DO and set a unit intervention cost for all variables. Performance metric We run all experiments for 10 replicates and show the average convergence path at every time step. We then compute the values of a modified gap metric3 across time steps and with standard errors across replicates. The metric is defined as Gt = y(x s,t) y(xinit) y y(xinit) + H H(x s,t) H where y( ) represents the evaluation of the objective function, y is the global minimum, and xinit and x s,t are the first and best evaluated point, respectively. The term H H(x s,t) H with H(x s,t) denoting the number of explorative trials needed to reach x s,t captures the speed of the optimization. This term is equal to zero when the algorithm is not converged and equal to (H 1)/H when the algorithm converges at the first trial. We have 0 Gt 1 with higher values denoting better performances. For each method we also show the average percentage of replicates where the optimal intervention set X s,t is identified. |
| Researcher Affiliation | Collaboration | Virginia Aglietti University of Warwick The Alan Turing Institute V.Aglietti@warwick.ac.uk Neil Dhir The Alan Turing Institute ndhir@turing.ac.uk Javier González Microsoft Research Cambridge Gonzalez.Javier@microsoft.com Theodoros Damoulas University of Warwick The Alan Turing Institute T.Damoulas@warwick.ac.uk |
| Pseudocode | Yes | Algorithm 1: DCBO |
| Open Source Code | Yes | 2A Python implementation is available at: https://github.com/neildhir/DCBO. |
| Open Datasets | Yes | Real-World Economic data (ECON.) We use DCBO to minimize the unemployment rate Ut of a closed economy... Time series data for 10 countries5 are used to construct a non-parametric simulator and to compute the causal prior for both DCBO and CBO...5Data were downloaded from https://www.data.oecd.org/ [Accessed: 01/04/2021]. All details in the supplement. |
| Dataset Splits | No | The paper does not explicitly provide details about training, validation, or test splits. It mentions using 'observational data' and 'interventional data' but not how these datasets are formally split for model training and evaluation. |
| Hardware Specification | No | The paper does not specify any hardware used for running the experiments. |
| Software Dependencies | No | The paper mentions 'A Python implementation is available' but does not specify any software dependencies with version numbers. |
| Experiment Setup | No | The paper discusses different experimental settings (STAT., NOISY, MISS., MULTIV., IND., NONSTAT., ECON., ODE) and compares performance using a modified gap metric, but it does not provide specific hyperparameter values, training configurations, or detailed system-level settings for reproducibility. |