Predicting Above-Sentence Discourse Structure Using Distant Supervision from Topic Segmentation
Authors: Patrick Huber, Linzi Xing, Giuseppe Carenini10794-10802
AAAI 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on two human-annotated discourse treebanks confirm that our proposal generates accurate tree structures on sentence and paragraph level, consistently outperforming previous distantly supervised models on the sentence-to-document task and occasionally reaching even higher scores on the sentence-to-paragraph level. |
| Researcher Affiliation | Collaboration | The University of British Columbia, Vancouver, Canada {huberpat, lzxing, carenini}@cs.ubc.ca This research was supported by the Language & Speech Innovation Lab of Cloud BU, Huawei Technologies Co., Ltd and the Natural Sciences and Engineering Research Council of Canada (NSERC). |
| Pseudocode | No | The paper includes a diagram (Figure 4) illustrating a greedy top-down approach and describes the algorithm in text, but it does not provide structured pseudocode or an algorithm block labeled as such. |
| Open Source Code | No | The paper includes a GitHub link (www.github.com/nlpat/MEGA-DT) in footnote 5, but this link refers to the MEGA-DT discourse treebank used for a baseline model (Huber and Carenini (2020b)), not the open-source code for the methodology described in this paper. |
| Open Datasets | Yes | We randomly sample a subset of Wikipedia articles from the Wikipedia dump3, strictly following the sampling scheme in Koshorek et al. (2018) (from here on called Wiki). Our Wiki corpus consists of 20, 000 documents... 3https://dumps.wikimedia.org/enwiki/latest/enwiki-latest-pages-articles.xml.bz2 We further train the topic segmentation model on the RST-DT (Carlson, Okurowski, and Marcu 2002) and GUM (Zeldes 2017) corpora. |
| Dataset Splits | Yes | We split our Wiki dataset into training, validation and test sets using the default 80%, 10%, 10% data-split. |
| Hardware Specification | No | The paper does not explicitly describe the hardware used to run its experiments. It does not mention any specific GPU models, CPU models, or other hardware specifications. |
| Software Dependencies | No | The paper mentions using a "BERT encoder" and a "hierarchical Bi LSTM network" but does not provide specific version numbers for these or any other software dependencies like Python, PyTorch, or TensorFlow versions. |
| Experiment Setup | Yes | To better understand and properly compare the performance of our discourse tree generation algorithm with previously published models, as well as a set of baselines, we evaluate all approaches on three partially overlapping discourse tree subsets from: sentence-to-paragraph (S-P), paragraph-to-document (P-D) and sentence-to-document (SD). In our evaluation, we find that distant supervision from topic segmentation achieves promising results on the highlevel tree structure generation task, consistently outperforming previous methods with distant supervision on sentenceto-document level and in some cases reaching superior performance compared to supervised models. Furthermore, complete (EDUto-document) gold tree structures need to be trimmed to: (1) only contain nodes covering a complete unit of the lowerbound of measure (e.g., sentences in S-D/S-P) and (2) do not contain any nodes covering multiple units of the upperbound of measure (e.g., paragraphs in S-P). Hence, we propose to trim and restrict discourse trees with the following two strategies: Trimming Trees to the Lower-Bound: ... Restricting Trees to the Upper-Bound: ... |