PinNet: Pinpoint Instructive Information for Retrieval Augmented Code-to-Text Generation
Authors: Han Fu, Jian Tan, Pinhan Zhang, Feifei Li, Jianling Sun
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Using extensive experiments on code summarization and SQL-to-text generation, we demonstrate that the proposed method can significantly outperform all of the baselines. |
| Researcher Affiliation | Collaboration | 1Alibaba Group, Hangzhou, China 2College of Computer Science and Technology, Zhejiang University, Hangzhou, China. |
| Pseudocode | No | The paper describes algorithms and components but does not provide structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide a direct link or explicit statement about the availability of its own open-source code. |
| Open Datasets | Yes | For code summarization task, we use a widely adopted benchmark, Code XGLUE (Lu et al., 2021)... For SQL-to-text generation task, we conduct experiments on two datasets, Wiki SQL (Zhong et al., 2017) and Stack Overflow (Iyer et al., 2016). |
| Dataset Splits | Yes | Table 9. Statistics of training/developing/testing samples and retrieval databases of different datasets. Code XGLEU-Java Train 164,923 Dev 5,183 Test 10,955. |
| Hardware Specification | Yes | The models are trained on a single Nvidia A100 GPU. |
| Software Dependencies | No | The paper mentions pre-trained models like Graph Code BERT and RoBERTa, and optimizers like Adam, but does not provide specific version numbers for software dependencies such as deep learning frameworks or libraries. |
| Experiment Setup | Yes | For both Code XGLEU-Java and Code XGLEU-Python datasets, we truncate the input and output sequence to maximum 512 and 80 tokens, respectively. ... The Pin Net-Ret model is trained with Adam (Kingma & Ba, 2014) using batch size 128. The learning rate is 2 10 5. To train the generation model sufficiently, we set the learning rates of Pin Net-Enc and Pin Net-Dec as 1 10 5 and 1 10 4 , respectively. The generation model is trained with batch size 64. For the retrieval task, we set m1 and m2 to 0.2 and 0.4, respectively. The hyper-parameters of the generation loss, α and β, are both set to 1.0. During training, we use Pin Net-Ret to select the top-10 code summaries by cosine score. At runtime, we use beam search and the beam size is set to 4. |