Automatic Fact-Guided Sentence Modification
Authors: Darsh Shah, Tal Schuster, Regina Barzilay8791-8798
AAAI 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Applied to a Wikipedia fact update dataset, our method successfully generates updated sentences for new claims, achieving the highest SARI score. Furthermore, we demonstrate that generating synthetic data through such rewritten sentences can successfully augment the FEVER fact-checking training dataset, leading to a relative error reduction of 13%. |
| Researcher Affiliation | Academia | Massachusetts Institute of Technology {darsh, tals, regina}@csail.mit.edu |
| Pseudocode | No | The paper describes its models and methods in detail using natural language and mathematical formulas, but it does not include any pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | Yes | 1Code: (1) https://github.com/Tal Schuster/Token Masker (2) https://github.com/darsh10/split encoder pointer summarizer |
| Open Datasets | Yes | Training Data from FEVER We use FEVER (Thorne et al. 2018), the largest available Wikipedia based fact-checking dataset to train our models for both of our tasks. |
| Dataset Splits | Yes | This dataset (version 0.2) contains 531 claim-evidence pairs for validation and 534 claim-evidence pairs for testing. |
| Hardware Specification | No | The paper does not specify the exact hardware used for running experiments, such as specific GPU or CPU models. It mentions using Bi LSTM encoders and BERT, which implies computational resources, but no explicit details are provided. |
| Software Dependencies | No | The paper mentions using the Allen NLP framework (Gardner et al. 2018), GloVe (Pennington, Socher, and Manning 2014) word embeddings, and a BERT (Devlin et al. 2018) classifier. However, specific version numbers for these software components are not provided. |
| Experiment Setup | Yes | We use Bi LSTM (Sak, Senior, and Beaufays 2014) encoders with hidden dimensions of 100 and share the parameters of the claim and original sentence encoders. The model is trained for up to 100 epochs with a patience value of 10...We use a one layer Bi LSTM for encoding and decoding with a hidden dimension of 256. The parameters of the two encoders are shared. The model is trained with batches of size 64 for a total of 50K steps. ...The model is fine-tuned for 3 epochs |