Towards Faithfulness in Open Domain Table-to-text Generation from an Entity-centric View
Authors: Tianyu Liu, Xin Zheng, Baobao Chang, Zhifang Sui13415-13423
AAAI 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The experiments show that the proposed augmented plan based methods not only reduce about 60% hallucinations compared with the counterparts trained in the end-to-end fashion in full dataset setting but also greatly enhance the model performance in few shot learning. We also achieve the state-of-the-art performance on WIKIPERSON dataset and show that the proposed methods work well on both pre-trained models and (non pre-trained) Transformer models. |
| Researcher Affiliation | Academia | Tianyu Liu 1 , Xin Zheng 2 3 , Baobao Chang 1 4, Zhifang Sui 1 4 1 Ministry of Education (MOE) Key Laboratory of Computational Linguistics, School of EECS, Peking University 2 Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences, Beijing, China 3 University of Chinese Academy of Sciences, Beijing, China 4 Pengcheng Laboratory, Shenzhen, China |
| Pseudocode | No | Not found. The paper does not contain structured pseudocode or algorithm blocks (e.g., clearly labeled algorithm sections or code-like formatted procedures). |
| Open Source Code | No | Our implementation for Transformer and BART is based on fairseq (Ott et al. 2019) 10. For PG-Net, our implementation is based on a pytorch version reimplementation on github11. The paper mentions existing tools/reimplementations but does not provide a specific link or statement that their own code for the described methodology is released or available. |
| Open Datasets | Yes | The WIKIPERSON dataset8 (Wang et al. 2018) contains 250186, 30487, and 29982 (table, text) pairs in training, dev and test sets respectively, which is exactly the same setting as Wang et al. (2018) and Wang et al. (2020b). (Footnote 8 links to https://github.com/EagleW/Describing_a_Knowledge_Base) |
| Dataset Splits | Yes | The WIKIPERSON dataset8 (Wang et al. 2018) contains 250186, 30487, and 29982 (table, text) pairs in training, dev and test sets respectively, which is exactly the same setting as Wang et al. (2018) and Wang et al. (2020b). |
| Hardware Specification | No | Not found. The paper does not provide specific hardware details (e.g., exact GPU/CPU models, processor types, or memory amounts) used for running its experiments. |
| Software Dependencies | No | Our implementation for Transformer and BART is based on fairseq (Ott et al. 2019) 10. For PG-Net, our implementation is based on a pytorch version reimplementation on github11. We use Stanza (Qi et al. 2020) toolkit throughout our experiments. However, specific version numbers for these software packages or any other dependencies are not provided. |
| Experiment Setup | Yes | For Transformer model, we use the same parameter setting as Wang et al. (2020b) (with copy mechanism). The hidden units of the multi-head attention and the feed-forward layer are set to 2048. The embedding size is set to 512. The number of heads is set to 8, and the number of Transformer blocks is 3. Beam size is set to be 5. For other parameters except learning rate, we use the default setting in fairseq according to Vaswani et al. (2017). For BART, we use the pretrained BART (large) architecture in fairseq which has 12 layers. For PG-Net, we set the embedding size, hidden size and beam size as 256, 512 and 5 respectively. For all three models, we use adam optimizer (Kingma and Ba 2014) (β1 = 0.9, β2 = 0.999). The learning rates are set as 1e-5, 1e-5, 3e-5 for PG-Net, Transformer and BART models respectively. The vocabulary size is limited to 50k for every model. |