Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Beyond Higher Rank: Token-wise Input-Output Projections for Efficient Low-Rank Adaptation
Authors: Shiwei Li, Xiandi Luo, Haozhao Wang, Xing Tang, Ziqiang Cui, Dugang Liu, Yuhua Li, Xiuqiang He, Ruixuan Li
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments across multiple models and datasets demonstrate that Top Lo RA consistently outperforms Lo RA and its variants. |
| Researcher Affiliation | Academia | 1Huazhong University of Science and Technology 2Shenzhen Technology University 3City University of Hong Kong 4Shenzhen University EMAIL EMAIL, EMAIL EMAIL, EMAIL |
| Pseudocode | No | The paper describes its methodology using mathematical equations and textual explanations, but it does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | The code is available at https://github.com/Leopold1423/toplora-neurips25. |
| Open Datasets | Yes | First, we assess Top Lo RA’s natural language understanding (NLU) capabilities on the GLUE benchmark [42], which includes eight sub-tasks. ... Next, we examine Top Lo RA’s natural language generation (NLG) capabilities on two reasoning benchmarks compiled by Hu et al. [11]: mathematical reasoning and commonsense reasoning |
| Dataset Splits | Yes | The GLUE benchmark [42] includes two single-sentence classification tasks (Co LA, SST-2), five pairwise text classification tasks (MNLI, RTE, QQP, MRPC, and QNLI), and one text similarity prediction task (STS-B). ... Mathematical and commonsense reasoning tasks contain 10K and 170K training samples, respectively, along with several test tasks. Note that we directly utilize the data from [11] for our experiments. |
| Hardware Specification | No | The computation is completed in the HPC Platform of Huazhong University of Science and Technology. |
| Software Dependencies | No | Adam W [29] is used with β1 = 0.9, β2 = 0.999, ϵ = 1e 8, and no weight decay. ... The learning rate is selected from the set {3e 5, 1e 4, 3e 4, 1e 3}, with optimal values of 3e 4 for Ro BERTa-Base and 1e 4 for Ro BERTa-Large. |
| Experiment Setup | Yes | The learning rates for Ro BERTa-Base and Ro BERTa-Large were set to 3e-4 and 1e-4, respectively. A warm-up ratio of 0.03 and linear learning rate decay were used. The number of training epochs varied across sub-tasks; further details are provided in Appendix A. ... The scaling factor is set to α = 2r, where r is the Lo RA rank. Lo RA is applied to the query and value weights with a dropout rate of 0.05, using full precision (FP32). ... Adam W [29] is used with β1 = 0.9, β2 = 0.999, ϵ = 1e 8, and no weight decay. ... A warm-up of 100 steps is applied, and the batch size is set to 16. The maximum sequence length is 256. |