Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Efficient Online Estimation of Causal Effects by Deciding What to Observe
Authors: Shantanu Gupta, Zachary Lipton, David Childers
NeurIPS 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, we validate our findings experimentally 1 (Section 6). We validate our methods on synthetic data generated from known causal graphs (see Appendix D for parameter values and moment conditions used). |
| Researcher Affiliation | Academia | Shantanu Gupta, Zachary C. Lipton, David Childers Carnegie Mellon University EMAIL |
| Pseudocode | Yes | Figure 1: Algorithms for OMS-ETC and OMS-ETG. |
| Open Source Code | Yes | The code and data are available at https://www.github.com/acmi-lab/ online-moment-selection. |
| Open Datasets | Yes | Finally, we demonstrate the effectiveness of our methods on two semi-synthetic datasets: the Infant Health Development Program (IHDP) dataset [19] and a Vietnam era draft lottery dataset [2]. Replication data for: Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records, 2009. URL https://doi.org/10. 7910/DVN/PLF0YL. |
| Dataset Splits | No | No specific train/validation/test dataset splits (e.g., percentages, sample counts, or explicit standard splits) are mentioned. The paper describes generating semi-synthetic data and states 'The MSE values are computed across 12, 000 runs.' |
| Hardware Specification | No | No specific hardware details (e.g., GPU/CPU models, processor types, memory amounts) used for running experiments are mentioned in the paper. |
| Software Dependencies | No | No specific software dependencies with version numbers (e.g., library names like 'PyTorch 1.9' or solver versions like 'CPLEX 12.4') are provided. The paper only mentions using a 'regularized GMM weight matrix' and implies Python use given the GitHub link. |
| Experiment Setup | No | The paper mentions 'For OMS-ETG, we use the regularized GMM weight matrix with λW := 0.01' and 'The MSE values are computed across 12, 000 runs'. However, it does not provide specific hyperparameter values for model training (e.g., learning rates, batch sizes, epochs) or detailed system-level configurations. |