Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Spectral Adapter: Fine-Tuning in Spectral Space
Authors: Fangzhao Zhang, Mert Pilanci
NeurIPS 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We show through extensive experiments that the proposed fine-tuning model enables better parameter efficiency and tuning performance as well as benefits multi-adapter fusion. |
| Researcher Affiliation | Academia | Fangzhao Zhang Electrical Engineering Stanford University EMAIL Mert Pilanci Electrical Engineering Stanford University EMAIL |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code is released at https://github.com/pilancilab/spectral_adapter. |
| Open Datasets | Yes | Llama3 8B model with Orca Math dataset [38] and evaluation score on GSM8K benchmark [8]. |
| Dataset Splits | No | The paper mentions training data (Orca Math) and test data (GSM8K) but does not provide explicit training/validation/test splits with percentages or sample counts for all experiments, nor does it specify the method for creating such splits beyond "10K Orca Math data (shuffled)". |
| Hardware Specification | Yes | All experiments are done with NVIDIA RTX A6000 GPU. |
| Software Dependencies | No | The paper mentions software like Pytorch [44] and Hugging Face PEFT library but does not provide specific version numbers for these or other software dependencies. |
| Experiment Setup | Yes | Table 4 shows the hyperparameter setting for our Spectral Adapter A used for fine-tuning De BERTa V3base model in Section 4.1. We set number of diagonal blocks to be 4 and enable block sharing for OFT to maintain similar amount of trainable parameters. |