Externally Valid Policy Evaluation from Randomized Trials Using Additional Observational Data
Authors: Sofia Ek, Dave Zachariah
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The certified policy evaluations are illustrated using both simulated and real data. ... We will use both synthetic and real-world data to illustrate the main concepts of policy evaluation with limit curves (α, ℓΓ α). |
| Researcher Affiliation | Academia | Sofia Ek Uppsala University sofia.ek@it.uu.se Dave Zachariah Uppsala University dave.zachariah@it.uu.se |
| Pseudocode | Yes | Algorithm 1 A set of limit curves for policy π |
| Open Source Code | Yes | the code used for the experiments can be accessed here https://github.com/sofiaek/policy-evaluation-rct. |
| Open Datasets | Yes | To illustrate the application of policy evaluation with real data, we study the impact of seafood consumption on blood mercury levels with data from the 2013-2014 National Health and Nutrition Examination Survey (NHANES). Following Zhao et al. (2019)... Centers for Disease Control and Prevention (CDC). National Center for Health Statistics (NCHS). National Health and Nutrition Examination Survey Data. Hyattsville, MD: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, 2013. URL https: //wwwn.cdc.gov/nchs/nhanes/continuousnhanes/default.aspx?Begin Year=2013. |
| Dataset Splits | Yes | The hyperparameters were selected through a random search involving 200 runs, employing 5-fold cross-validation with the F1 score as the optimization metric. |
| Hardware Specification | Yes | All experiments were performed on a laptop with the following specifications: Intel Core i7-8650 CPU @ 1.9GHz, 16 GB DDR4 RAM, and Windows 10 Pro 64-bit operating system. The experiments utilized only the CPU. The total time required to run all the experiments was approximately half an hour. |
| Software Dependencies | Yes | All experiments were conducted using Version 1.7 of the Python implementation of XGBoost (Apache2.0 License). |
| Experiment Setup | Yes | A comprehensive list of hyperparameters is available in Table 2 for the synthetic case and Table 3 for the NHANES case. ... (Table 2: Parameter Value n_estimators 100 max_depth 2 learning_rate 0.05 objective binary:logistic min_child_weight 1 subsample 0.6 colsample_bytree 0.8 colsample_bylevel 0.4 scale_pos_weight ns=0/ns=1) |