Fact-Aware Sentence Split and Rephrase with Permutation Invariant Training
Authors: Yinuo Guo, Tao Ge, Furu Wei7855-7862
AAAI 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on the Web Split-v1.0 benchmark dataset show that our approaches can largely improve the performance over the previous seq2seq learning approaches. Moreover, an extrinsic evaluation on oie-benchmark verifies the effectiveness of our approaches by an observation that splitting long sentences with our state-of-the-art model as preprocessing is helpful for improving Open IE performance. We conduct experiments on the Web Split-v1.0 corpus |
| Researcher Affiliation | Collaboration | Yinuo Guo,1 Tao Ge,2 Furu Wei2 1Key Laboratory of Computational Linguistics School of EECS, Peking University, 2Microsoft Research Asia |
| Pseudocode | No | The paper does not contain any pseudocode or algorithm blocks. Figure 2 is an architectural diagram, not pseudocode. |
| Open Source Code | No | The paper mentions links to external resources for evaluation scripts and benchmarks (e.g., 'https://github.com/roeeaharoni/sprp-acl2018' for BLEU calculation and 'https://github.com/gabriel Stanovsky/oie-benchmark' for oie-benchmark evaluation), but does not provide a link or explicit statement about releasing the source code for their own proposed methodology. |
| Open Datasets | Yes | We conduct experiments on the Web Split-v1.0 corpus, which is a benchmark dataset to compare the performance of models for Sentence Split and Rephrase task. Its training set contains approximately 1.3 million C-S pairs; while its validation and test sets contain about 4,000 complex sentences, each of which is equipped with multiple references. For Fact-aware Sentence Encoding with multi-task learning, we construct a fact classification dataset based on the method mentioned in Section 3.1. The resulting dataset contains 94,930 training samples where half are positive and the other are negative samples constructed by fact corruption. To further verify the performance of our methods on large-scale datasets, we incorporate the Wiki Split (Botha et al. 2018) data for pre-training. |
| Dataset Splits | Yes | Its training set contains approximately 1.3 million C-S pairs; while its validation and test sets contain about 4,000 complex sentences, each of which is equipped with multiple references. |
| Hardware Specification | No | The paper describes model architectures (e.g., '1-layer LSTM (512 hidden units) encoder-decoder model', 'CNN layer', 'base Transformer'), but does not specify the hardware used for training or inference (e.g., CPU, GPU models, memory, or cloud instances). |
| Software Dependencies | No | The paper mentions software components and tools like '1-layer LSTM', 'Adam optimizer', 'CNN layer', 'beam search', 'byte-pair-encoding', and 'Transformer', but does not provide specific version numbers for these or any underlying programming languages/libraries (e.g., Python, PyTorch, TensorFlow versions) to ensure reproducibility of the software environment. |
| Experiment Setup | Yes | We use the Adam optimizer with a learning rate of 0.0005 with 8000 warmup steps. The training process lasts 30 epochs with the batch size of 64. During inference, the beam size is set to 12. For fact classification, as illustrated in Figure 2, we first use a CNN layer with Relu activation and max-pooling to generate a fixed length vector h s from hs, then concatenate h s with the fact vector hf encoded by a fact encoder Ef which is also a CNN layer with the same configuration: filter size n = 3, 4, 5 and filter number 24 for each size. |