Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation
Authors: Pengcheng Huang, Zhenghao Liu, Yukun Yan, Haiyan Zhao, Xiaoyuan Yi, Hao Chen, Zhiyuan Liu, Maosong Sun, Tong Xiao, Ge Yu, Chenyan Xiong
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results show that Param Mute significantly enhances faithfulness across both Co Faithful QA and the established Con Fi QA benchmark, achieving substantial reductions in reliance on parametric memory. These findings underscore the importance of mitigating internal knowledge dominance and provide a new direction for improving LLM trustworthiness in RAG. All codes are available at https://github.com/Open BMB/Param Mute. We conduct ablation studies to analyze the effectiveness of Param Mute s suppression strategy and to evaluate the contributions of its key components. |
| Researcher Affiliation | Collaboration | 1School of Computer Science and Engineering, Northeastern University, China 2Department of Computer Science and Technology, Institute for AI, Tsinghua University, China 3Microsoft Research Asia, Beijing, China 4Language Technologies Institute, Carnegie Mellon University, United States |
| Pseudocode | No | The paper describes methods and equations like FFN(xl i) = (σ(Klxl i)) V l and equations for activation ratio and loss functions, but there are no structured pseudocode or algorithm blocks presented in the main text or appendices. |
| Open Source Code | Yes | All codes are available at https://github.com/Open BMB/Param Mute. |
| Open Datasets | Yes | Our Co Faithful QA is constructed from six widely-used open-domain QA datasets: Natural Questions (NQ) [33], SQu AD [51], News QA [60], Trivia QA [31], Search QA [13], and Hotpot QA [69]. Natural Questions (CC BY-SA 3.0), News QA (MIT License), Search QA (Apache License 2.0), Trivia QA (Apache License 2.0), Hotpot QA (CC BY-SA 4.0), and SQu AD (CC BY-SA 4.0). |
| Dataset Splits | Yes | During the training stage of Param Mute, we construct the training data by randomly sampling 32,580 instances from the combined training sets of the six sub-datasets included in our benchmark, all of which are derived from the MRQA 2019 benchmark [18]. For comparative analysis, we partition D into a faithful subset D+ and an unfaithful subset D based on the faithfulness label yf. We then analyze the FFN activation patterns of the LLa MA3-8B-Instruct model across the two groups to investigate how activation behavior differs between faithful and unfaithful generations. |
| Hardware Specification | No | The paper states: 'To ensure a fair comparison, we use LLa MA3-8B-Instruct as the backbone model for all methods throughout our experiments.' However, no specific details about the hardware (e.g., GPU model, CPU type, memory) used for running the experiments are provided. |
| Software Dependencies | No | The paper mentions using LLa MA3-8B-Instruct as the backbone model and implementing Param Mute with Lo RA [24]. It also refers to the lm-evaluation-harness [19] for evaluation. However, specific version numbers for software dependencies such as Python, PyTorch, or other libraries are not explicitly provided. |
| Experiment Setup | Yes | In this work, all models are trained for 2,100 steps with a total batch size of 32 and a learning rate of 1e-4. To enhance training efficiency, we implement Param Mute with Lo RA [24]. For Param Mute, we set the number of suppressed UA-FFNs layers to N = 8, and the suppression coefficient in Eq. 11 is fixed at 0.0. The hyperparameters α and β, which control the relative contributions of LKAT and LKPO in Eq.8, are both set to 0.5. Additionally, we adopt a dynamic γ in LKPO (Eq. 10), which linearly transitions from an initial margin (γ0 = 1) to a final margin (γ = 5) as training progresses. |