Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
F-Adapter: Frequency-Adaptive Parameter-Efficient Fine-Tuning in Scientific Machine Learning
Authors: Hangwei Zhang, Chun Kang, Yan Wang, Difan Zou
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct the first systematic study of PEFT for pre-trained Large Operator Models (LOMs) obtained by scaling variants of Fourier Neural Operator. First, we observe that the widely used Low-Rank Adaptation (Lo RA) yields markedly poorer performance on LOMs than Adapter tuning. Then, we further theoretically establish that stacked Lo RA incurs a depthamplified lower bound on approximation error within Fourier layers, whereas adapters retain universal approximation capacity and, by concentrating parameters on energy-dominant low-frequency modes, attain exponentially decaying error with bottleneck width in the Fourier domain. Motivated by the robust empirical gains of adapters and by our theoretical characterization of PDE solutions as spectrally sparse, we introduce Frequency-Adaptive Adapter (F-Adapter). FAdapter allocates adapter capacity based on spectral complexity, assigning higher-dimension modules to low-frequency components and lower-dimension modules to high-frequency components. Our F-Adapters establish state-of-the-art (SOTA) results on multiple challenging 3D Navier Stokes benchmarks, markedly enhancing both generalization and spectral fidelity over Lo RA and other PEFT techniques commonly used in LLMs. In this work, we present the first systematic study of PEFT for pretrained LOMs. Through a combination of empirical analysis and theoretical investigation... |
| Researcher Affiliation | Academia | 1 School of Computing and Data Science, The University of Hong Kong 2 Institute for AI Industry Research, Tsinghua University 3 Beihang University |
| Pseudocode | Yes | Algorithm 1 F-Adapter PEFT Forward Pass in DPOT s [14] Fourier Attention Layer |
| Open Source Code | No | The code is publicly available at here. (from abstract) and The PDEBench datasets are publicly available and the code will be made publicly available upon acceptance. (from NeurIPS checklist justification) |
| Open Datasets | Yes | We fine-tune this model on two 3D Navier Stokes datasets from PDEBench [46] with standard parameter-efficient methods... (from Section 3.1) and The PDEBench datasets are publicly available and the code will be made publicly available upon acceptance. (from NeurIPS checklist justification) |
| Dataset Splits | Yes | To evaluate fine-tuning methods for LOMs, we focus on the threedimensional forecasting problem... We fine-tune this model on two 3D Navier Stokes datasets from PDEBench [46]... These datasets are configured with random initial conditions at M = 1.0 and M = 0.1. (from Section 3.1). Following the original protocol, we reserve 90 trajectories for training/validation and 10 for held-out testing within each subset. We utilize stratified sampling to preserve the subset ratios. (from Section C.6) |
| Hardware Specification | Yes | All experiments are conducted on a single NVIDIA A800 80 GB GPU. |
| Software Dependencies | No | During fine-tuning, we use the Adam W optimizer and train the model for 500 epochs with different efficiency levels (e.g., ranks for Lo RA and bottleneck dimensions for Adapter). (from Section 3.1). The paper does not provide specific version numbers for software libraries or environments. |
| Experiment Setup | Yes | During fine-tuning, we use the Adam W optimizer and train the model for 500 epochs with different efficiency levels (e.g., ranks for Lo RA and bottleneck dimensions for Adapter). The performance is evaluated on the test set using the L2 relative error (L2RE), a standard metric in operator learning [23]. All experiments are conducted on a single NVIDIA A800 80 GB GPU. Complete experimental details are provided in Appendix C. (from Section 3.1) We adopt zero-initialization for every Wup b and bup b so that, at the start of fine-tuning, the adapted path is an exact identity and does not perturb the pre-trained backbone. Down-projection weights are initialized with Kaiming-uniform initialization. (from section 4) |