Causal Estimation for Text Data with (Apparent) Overlap Violations
Authors: Lin Gui, Victor Veitch
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirical results show strong improvements in bias and uncertainty quantification relative to the natural baseline. Code, demo data and a tutorial are available at https://github.com/gl-ybnbxb/TI-estimator." and "5 EXPERIMENTS We empirically study the method s capability to provide accurate causal estimates with good uncertainty quantification Testing using semi-synthetic data (where ground truth causal effects are known), we find that the estimation procedure yields accurate causal estimates and confidence intervals. |
| Researcher Affiliation | Collaboration | Lin Gui1 and Victor Veitch1,2 1The University of Chicago 2Google Research |
| Pseudocode | No | The paper describes the estimation pipeline in detail (e.g., "We describe the three steps in detail.") but does not include any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code, demo data and a tutorial are available at https://github.com/gl-ybnbxb/TI-estimator. |
| Open Datasets | No | We use publicly available Amazon reviews for music products as the basis for our semi-synthetic data." (No specific link, DOI, or formal citation for the Amazon reviews dataset used, nor for the generated semi-synthetic data). |
| Dataset Splits | Yes | The model is trained in the k-folding fashion with 5 folds." and "The maximum number of epochs is set as 20, with early stopping based on validation loss with a patience of 6. |
| Hardware Specification | No | The paper mentions 'We acknowledge the University of Chicago s Research Computing Center for providing computing resources.' but does not specify any particular hardware details such as GPU models, CPU types, or memory. |
| Software Dependencies | No | For the language model, we use the pretrained distilbert-base-uncased model provided by the transformers package. The propensity model is implemented by running the Gaussian process regression using Gaussian Process Classifier in the sklearn package" (No specific version numbers for transformers or sklearn packages). |
| Experiment Setup | Yes | We apply the Adam optimizer (Kingma and Ba, 2014) with a learning rate of 2e 5 and a batch size of 64. The maximum number of epochs is set as 20, with early stopping based on validation loss with a patience of 6. Each experiment is replicated with five different seeds and the final ˆQ(a, xi) predictions are obtained by averaging the predictions from the 5 resulting models. |