KID-Review: Knowledge-Guided Scientific Review Generation with Oracle Pre-training
Authors: Weizhe Yuan, Pengfei Liu11639-11647
AAAI 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimentally, we perform a comprehensive evaluation (human and automatic) from different perspectives. Empirical results have shown the effectiveness of different types of knowledge as well as oracle pre-training. |
| Researcher Affiliation | Academia | Weizhe Yuan, Pengfei Liu Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA, USA {weizhey, pliu3}@cs.cmu.edu |
| Pseudocode | No | The paper describes the model architecture and processes using text and diagrams (Fig. 2, 3, 4) and mathematical equations, but does not include explicit pseudocode or algorithm blocks. |
| Open Source Code | Yes | We make all code, relevant datasets available: https://github.com/yyy-Apple/KIDReview as well as the KIDREVIEW system: http://nlpeer.reviews. |
| Open Datasets | Yes | We use ASAP-Review dataset introduced by Yuan, Liu, and Neubig (2021) for our experiment. It consists of ICLR papers from 2017-2020 and Neur IPS papers from 2016-2019, together with their aligned reviews. |
| Dataset Splits | Yes | To make a fair comparison, we use the same training, validation, and test split as them. The basic statistics of this dataset are shown in Tab. 1. Figure 5: Train Validation Test 8,000 6,993 Unique papers Aligned papers |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU/CPU models, memory) used for running experiments. |
| Software Dependencies | No | The paper mentions using BART (Lewis et al. 2020) and a specific BART checkpoint ('bart-large-cnn') but does not provide version numbers for BART or any other key software dependencies. |
| Experiment Setup | Yes | We set the embedding size to be 128 when learning citation embeddings. We use two GAT layers for the concept graph, each with 4 attention heads, and we set the hidden size to be 200. To get the initial concept graph embeddings, we set l = Nenc/2, where Nenc denotes the total number of layers in BART encoder. For each BART decoder layer, we add another cross-attention module to attend to entity node representations on top of the regular cross attention module. We use beam search decoding during generation and adopt the same parameters following Yuan, Liu, and Neubig (2021) for all systems. |