Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
High Precision Causal Model Evaluation with Conditional Randomization
Authors: Chao Ma, Cheng Zhang
NeurIPS 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirical studies demonstrate the improved of our estimator, highlighting its potential on achieving near-RCT performance. |
| Researcher Affiliation | Industry | Chao Ma Microsoft Research Cambridge, UK EMAIL Cheng Zhang Microsoft Research Cambridge, UK EMAIL |
| Pseudocode | No | The paper does not contain any pseudocode or explicitly labeled algorithm blocks. |
| Open Source Code | No | The paper states that 'All methods are implemented via Econ ML package' and provides a link to the Econ ML GitHub repository, but it does not state that the code for the methodology described in *this* paper is open source or publicly available. |
| Open Datasets | No | The paper uses 'synthetic datasets' (csuite datasets) generated based on structural causal models and also 'synthetic observational data' generated via a described process. It does not use or provide access information for a pre-existing publicly available dataset. |
| Dataset Splits | No | The paper does not explicitly provide details about training, validation, and test dataset splits with specific percentages, counts, or explicit statements about how data was partitioned for validation. |
| Hardware Specification | No | The paper does not provide any specific hardware details such as GPU or CPU models, or cloud computing specifications used for running the experiments. |
| Software Dependencies | No | The paper mentions 'All methods are implemented via Econ ML package' but does not provide specific version numbers for this package or any other software dependencies. |
| Experiment Setup | Yes | The paper details experimental settings such as varying the degree of imbalance (σ2 β = 1, 5, 10), the dimensionality of covariates (nx = 30, nw = 30), and repeating simulations '100 times'. |