Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Beyond What If: Advancing Counterfactual Text Generation with Structural Causal Modeling
Authors: Ziao Wang, Xiaofeng Zhang, Hongwei Du
IJCAI 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments have been conducted on both a public story generation dataset and a specially constructed dataset in the financial domain. The experimental results demonstrate that our approach achieves state-of-the-art performance across a range of automatic and human evaluation criteria, underscoring its effectiveness and versatility in diverse text generation contexts. |
| Researcher Affiliation | Academia | Ziao Wang , Xiaofeng Zhang , Hongwei Du School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, China EMAIL, EMAIL |
| Pseudocode | No | The paper includes architectural diagrams (Figure 2) and mathematical formulations but no structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any explicit statement about making its source code publicly available or a link to a code repository. |
| Open Datasets | Yes | We use publicly available counterfactual story generation dataset [Qin et al., 2019] and our constructed counterfactual financial text generation dataset to valid our proposed method. |
| Dataset Splits | No | The paper mentions a 'testing set' for human evaluation, but does not provide specific train/validation/test dataset splits (percentages or counts) in the main text. It defers dataset details and parameter settings to the appendix: 'The detail of the story dataset and the parameter settings are provided in the appendix B due to page limit.' |
| Hardware Specification | No | The paper does not specify any hardware details (e.g., GPU/CPU models, memory, cloud instances) used for running the experiments. |
| Software Dependencies | No | The paper does not provide specific version numbers for any software or libraries used in the experiments. |
| Experiment Setup | No | The paper states, 'The detail of the story dataset and the parameter settings are provided in the appendix B due to page limit,' indicating that such information is not present in the main body of the paper. |