Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
A Linear Algebraic Framework for Counterfactual Generation
Authors: Jong-Hoon Ahn, Akshay Vashist
ICLR 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Using simulated LDL cholesterol datasets, we show that our method significantly outperforms the most cited methods of counterfactual generation. |
| Researcher Affiliation | Industry | Jong-Hoon Ahn & Akshay Vashist Otsuka Pharmaceutical Development & Commercialization, Inc. Princeton, NJ 08540, USA EMAIL |
| Pseudocode | No | The paper provides mathematical derivations and equations but does not include explicit pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain any explicit statement about releasing the source code or a link to a code repository for the proposed methodology. |
| Open Datasets | Yes | To evaluate our counterfactual generation method provided by Theorem 2.3, we used the simulated LDL cholesterol dataset that had been used to evaluate the Sync Twin algorithm (Qian et al., 2021). |
| Dataset Splits | No | The paper mentions training data sizes (e.g., 'a dataset of N0 = N1 = 200 and a dataset of N0 = 1000 and N1 = 200') and refers to a 'test dataset', but it does not explicitly provide details for training, validation, and test dataset splits or percentages. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU/CPU models, memory) used to run the experiments. |
| Software Dependencies | No | The paper does not list specific software dependencies with version numbers (e.g., 'Python 3.8, PyTorch 1.9'). |
| Experiment Setup | Yes | To produce Table 1 as a reproduction of Table 2 of the Sync Twin paper (Qian et al., 2021), we trained our model with K0 = K1 = 2 and M = 85 from a dataset of N0 = N1 = 200 and a dataset of N0 = 1000 and N1 = 200. We also introduced confounding bias denoted by pn for the n-th patient: pn = p0 for ωn = 0 and pn = 1 for ωn = 1. The constant p0 controls the degree of confounding bias (smaller p0, larger bias). |