Neural Sentence Ordering Based on Constraint Graphs
Authors: Yutao Zhu, Kun Zhou, Jian-Yun Nie, Shengchao Liu, Zhicheng Dou14656-14664
AAAI 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments on five benchmark datasets show that our method outperforms all existing baselines significantly, achieving a new state-of-the-art performance. The results demonstrate the advantage of considering multiple types of order information and using graph neural networks to integrate sentence content and order information for the task. Our code is available at https://github.com/Dao D/ Constraint Graph4NSO. |
| Researcher Affiliation | Academia | Yutao Zhu1, Kun Zhou2, Jian-Yun Nie1, Shengchao Liu1,3, Zhicheng Dou4 1 Universit e de Montr eal, Montr eal, Qu ebec, Canada 2 School of Information, Renmin University of China, Beijing, China 3 Mila Qu ebec Artificial Intelligence Institute, Montr eal, Canada 4 Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China yutao.zhu@umontreal.ca, francis kun zhou@163.com, nie@iro.umontreal.ca, liusheng@mila.quebec, dou@ruc.edu.cn |
| Pseudocode | No | The paper describes its methodology using text and mathematical equations but does not include a formally labeled pseudocode block or algorithm section. |
| Open Source Code | Yes | Our code is available at https://github.com/Dao D/ Constraint Graph4NSO. |
| Open Datasets | Yes | Following previous work (Cui et al. 2018; Kumar et al. 2020), we conduct experiments on five public datasets. The detailed statistics of these datasets are shown in Table 2. Neur IPS/AAN/NSF abstracts (Logeswaran, Lee, and Radev 2018). SIND captions (Huang et al. 2016). ROCStory (Mostafazadeh et al. 2016). |
| Dataset Splits | Yes | The data are split into training, validation, and test set according to the publication year. Following (Wang and Wan 2019), we make an 8:1:1 random split on the dataset to get the training, validation, and test set. (See Table 2 for specific counts per split). |
| Hardware Specification | Yes | All models are implemented with Py Torch (Paszke et al. 2019) and trained on a TITAN V GPU. |
| Software Dependencies | No | The paper mentions software components like PyTorch (Paszke et al. 2019), BERT uncased model (Wolf et al. 2019), and Adam W (Loshchilov and Hutter 2019). However, it does not provide specific version numbers for these software libraries, which are necessary for full reproducibility. |
| Experiment Setup | Yes | In the first phase, we employ BERT uncased model (Wolf et al. 2019) with an MLP to predict constraints. The batch size and learning rate is set as 50 and 5e-5 respectively for all datasets. In the second phase, sentences are also represented by an uncased BERT model and a dropout layer with rate 0.1 is applied over the representations. Three GINs with the number of layers L = {2, 3, 5} are used for the three corresponding graphs (k = 3) obtained in the first phase (denoted as g1, g2 and g3 respectively). The hidden size of all GIN layers is 512 (m = 512). A Re LU activation function is added between each layer. ϵ is tuned in {0, 0.1, 0.5} and set as 0 according to the experimental results on validation set. Batch normalization is applied to avoid overfitting. The batch size is set as 128 for all datasets while the learning rate is set as 1e-4, 5e-4, 6e-4, 4e-4 and 3e-4 for Neur IPS, AAN, NSF, SIND and ROCStory dataset. The maximum number of words in sentences is set as 50 on all datasets, which means sentences containing more than 50 words are truncated while those having less than 50 words are padded. All paragraphs in the datasets are randomly shuffled. The models in both phases are optimized with Adam W (Loshchilov and Hutter 2019). |