Reliable Decision Support using Counterfactual Models
Authors: Peter Schulam, Suchi Saria
NeurIPS 2017 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the benefits of the CGP on two important decision-support tasks: risk prediction and what if? reasoning for individualized treatment planning.We demonstrate the CGP on two decision-support tasks. First, we show how the CGP can make reliable risk predictions that do not depend on the action policy in the training data.On the other hand, we show that predictions made by models trained using classical supervised learning objectives are sensitive to the policies.In our second experiment, we use data from a real intensive care unit (ICU) to learn the CGP, and qualitatively demonstrate how the CGP can be used to compare counterfactuals and answer what if? questions, which could offer medical decision-makers a powerful new tool for individualized treatment planning. |
| Researcher Affiliation | Academia | Peter Schulam Department of Computer Science Johns Hopkins University Baltimore, MD 21211 pschulam@cs.jhu.edu Suchi Saria Department of Computer Science Johns Hopkins University Baltimore, MD 21211 ssaria@cs.jhu.edu |
| Pseudocode | No | The paper does not contain any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper mentions "simulation code included in the supplement" but does not provide a concrete access link or explicit statement about the general availability of the source code for the proposed Counterfactual Gaussian Process (CGP) methodology itself. |
| Open Datasets | Yes | We extract observational creatinine traces from the publicly available MIMIC-II database [Saeed et al., 2011]. |
| Dataset Splits | Yes | We shuffle the data and use 300 traces for training, 50 for validation and model selection, and 78 for testing. |
| Hardware Specification | No | The paper does not provide any specific hardware details such as GPU models, CPU models, or memory specifications used for running the experiments. |
| Software Dependencies | No | The paper mentions using BFGS for optimization, citing [Nocedal and Wright, 2006], but does not provide specific version numbers for any software libraries, programming languages, or other dependencies used in the implementation. |
| Experiment Setup | Yes | The covariance function is shared between all classes, and is defined using a Matérn 3/2 kernel (variance 0.22, lengthscale 8.0) and independent Gaussian noise (scale 0.1) added to each observation. For both the baseline GP and CGP, we use a mixture of three GPs (as was used to simulate the data). The mean for each class is zero (i.e. we assume there is no deviation from the initial value on average). We parameterize the covariance function using the sum of two non-stationary kernel functions. Let φ : t [1, t, t2] R3 denote the quadratic polynomial basis, then the first kernel is k1(t1, t2) = φ (t1)Σφ(t2), where Σ R3 3 is a positive-definite symmetric matrix parameterizing the kernel. The second kernel is the covariance function of the integrated Ornstein-Uhlenbeck (IOU) process (see e.g., Taylor et al. 1994), which is parameterized by two scalars α and ν and defined as k IOU(t1, t2) = ν2 2α3 2αmin(t1, t2) + e αt1 + e αt2 1 e α|t1 t2| . The two response functions are included in the mean function of the GP, and each class in the mixture has a unique set of response function parameters. We fit the CGP outcome model using Equation 3, and select the number of classes in the mixture using fit on the validation data (we choose three components). |