HLM-Cite: Hybrid Language Model Workflow for Text-based Scientific Citation Prediction
Authors: Qianyue Hao, Jingyang Fan, Fengli Xu, Jian Yuan, Yong Li
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate HLM-Cite on a dataset across 19 scientific fields, demonstrating a 17.6% performance improvement comparing SOTA methods. |
| Researcher Affiliation | Academia | Qianyue Hao, Jingyang Fan, Fengli Xu , Jian Yuan, Yong Li Department of Electronic Engineering, BNRist, Tsinghua University Beijing, China |
| Pseudocode | No | The paper illustrates the workflow in Figure 2, but it does not contain structured pseudocode or explicitly labeled algorithm blocks. |
| Open Source Code | Yes | Our code is open-source at https://github.com/tsinghua-fib-lab/H-LM for reproducibility. |
| Open Datasets | Yes | We conduct experiments based on Microsoft Academic Graph (MAG) [15], which archives hundreds of millions of research papers across 19 major scientific domains, forming a huge citation network. |
| Dataset Splits | No | The paper states, 'We randomly divide the sampled queries into 8:2 as training and testing sets.' It does not explicitly mention a separate validation set split or provide details on how a validation set was created or used for hyperparameter tuning separate from the train/test split. |
| Hardware Specification | Yes | The training process takes approximately 12 hours on 8 NVIDIA A100 80G GPUs in total. |
| Software Dependencies | Yes | OS Ubuntu 22.04.2 CUDA 11.7 Python 3.11.4 Pytorch 2.0.1 |
| Experiment Setup | Yes | We conduct the curriculum finetuning of our retrieval module with the batch size of 512 and 96 respectively in two stages, and each train for 10 epochs. The training process takes approximately 12 hours on 8 NVIDIA A100 80G GPUs in total. (Appendix A.2 provides further details on batch size, number of epochs, max token length, optimizer, learning rate, and random seed). |