Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
AdaMSS: Adaptive Multi-Subspace Approach for Parameter-Efficient Fine-Tuning
Authors: Jingjing Zheng, Wanglong Lu, Yiming Dong, Chaojie Ji, Yankai Cao, Zhouchen Lin
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments across image classification, natural language understanding, and natural language generation tasks show that Ada MSS achieves comparable performance to full fine-tuning and outperforms other parameter-efficient fine-tuning methods in most cases, all while requiring fewer trainable parameters. Notably, on the Vi T-Large model, Ada MSS achieves 4.7% higher average accuracy than Lo RA across seven tasks, using just 15.4% of the trainable parameters. On Ro BERTa-Large, Ada MSS outperforms Pi SSA by 7% in average accuracy across six tasks while reducing the number of trainable parameters by approximately 94.4%. These results demonstrate the effectiveness of Ada MSS in parameter-efficient fine-tuning. |
| Researcher Affiliation | Collaboration | Jingjing Zheng1,2,3,6 EMAIL Wanglong Lu4 EMAIL Yiming Dong1 EMAIL Chaojie Ji2,3,6 EMAIL Yankai Cao2,3,5, EMAIL Zhouchen Lin1,7,8, EMAIL 1State Key Lab of General AI, School of Intelligence Science and Technology, Peking University 2Institute of Applied Mathematics, The University of British Columbia 3Centre for AI Decision-Making and Action, The University of British Columbia 4AI Analytics Team, Nasdaq, St. John s, NL, Canada |
| Pseudocode | Yes | Algorithm 1 Initialization of Ada MSS Input: W 0, R, and rk. Output: Estimated K, {A(k)}K k=1, and the initialized values of {B(k)}K k=1 and {C(k)}K k=1. ... Algorithm 2 Estimation of K and Subspace Segmentation [16] Input: V 0, R, K0, and τ > 0. |
| Open Source Code | Yes | The code for Ada MSS is available at https: //github.com/jzheng20/Ada MSS. |
| Open Datasets | Yes | We evaluate all methods on IC using the widely adopted Vision Transformer (Vi T) [12], a prevalent foundation model in computer vision, across seven public datasets: Oxford Pets4, Stanford Cars3, CIFAR103, Euro SAT5, FGVC3, RESISC456, and CIFAR1003. |
| Dataset Splits | No | NLG: Following [2], we use the datasets listed in Table 21, and all experiments are conducted on 100K-example subsets and trained for a single epoch. The results are averaged over three runs. ... IC and NLU: The reported results are averaged over five random seeds. |
| Hardware Specification | No | The authors also gratefully acknowledge the computing resources and services provided by Digital Research Alliance of Canada (www.alliancecan.ca), and Advanced Research Computing at the University of British Columbia. |
| Software Dependencies | No | All experiments were performed with Python version 3.12.3. |
| Experiment Setup | Yes | A detailed list of all hyper-parameters and settings can be found in the Tables 18-20. In our experiments, we set R = 100, ρ = 3, and τ = 0.01. If not specified, K0 is set to 10. |