Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
QiMeng-SALV: Signal-Aware Learning for Verilog Code Generation
Authors: Yang Zhang, Rui Zhang, Jiaming Guo, Huang Lei, Di Huang, Yunpu Zhao, Shuyao Cheng, Pengwei Jin, Chongxiao Li, Zidong Du, Xing Hu, Qi Guo, Yunji Chen
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments demonstrate that our method achieves state-of-the-art performance on Verilog Eval and RTLLM, with a 7B parameter model matching the performance of the Deep Seek v3 671B model and significantly outperforming the leading open-source model Code V trained on the same dataset. Our code is available at https://github.com/Qi Meng-IPRC/Qi Meng SALV. |
| Researcher Affiliation | Academia | 1State Key Lab of Processors, Institute of Computing Technology, CAS 2University of Chinese Academy of Sciences 3University of Science and Technology of China |
| Pseudocode | No | The paper describes the methodology in narrative text and with diagrams (Figure 2) but does not include any explicit pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code is available at https://github.com/Qi Meng-IPRC/Qi Meng SALV. |
| Open Datasets | Yes | Our training dataset is sourced from Code V [4], consisting of 165k samples obtained by crawling all publicly available Verilog module code on Git Hub. |
| Dataset Splits | No | The paper uses a 135k sample dataset for fine-tuning but does not explicitly specify how this dataset is split into training, validation, and test sets for the model's own training process. It evaluates on external benchmarks (Verilog Eval, RTLLM) which have their own evaluation sets. |
| Hardware Specification | Yes | For the SFT phase, we execute full-parameter optimization across 2 training epochs utilizing a cluster of 4 NVIDIA A100-80GB SMX GPUs, with the complete training procedure consuming approximately 20 hours. Subsequently, during the Signal-aware DPO stage, we implement Lo RAbased fine-tuning over 7000 steps on an array of 8 NVIDIA A100-40GB GPUs, resulting in a total training duration of roughly 15 hours. |
| Software Dependencies | No | The paper mentions tools like Yosys [29] and the Adam [32] optimizer, but does not provide specific version numbers for these or other key software components or programming libraries used for implementation. |
| Experiment Setup | Yes | In training, we perform full-parameter fine-tuning for 2 epochs during the SFT, followed by about 1 epoch (7000 steps) of Lo RA-based[31] fine-tuning in the Signal-aware DPO. Optimization is carried out using the Adam [32] optimizer with a cosine annealing learning rate schedule, where the initial learning rates are set to 1.0e-5 for SFT and 5.0e-6 for Signal-aware DPO. The training configuration employs a global batch size of 64 and a maximum sequence length of 2048 tokens. |