Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Model Editing for Vision Transformers
Authors: Xinyi Huang, Kangfei Zhao, Long-Kai Huang
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments across multiple real-world datasets and model editing benchmarks demonstrate that our method consistently outperforms existing model editing methods for Vi Ts, achieving superior generalization and locality. Our experiments are designed to answer the following research questions: Q1: Does Refine Vi T, which edits the MSA modules, outperform state-of-the-art methods that primarily focus on editing the MLP modules, in terms of both editing success and locality in Vi Ts? (Sections 6.1) Q2: Can Refine Vi T generalize to other Vi T-based models such as CLIP-Vi T? (Section 6.2) Q3: Why does Refine Vi T work? Specifically, is it effective in identifying the attention-head-level features responsible for prediction failures? (Appendix C) Ablation studies and sensitivity analyses are provided in Appendix D. |
| Researcher Affiliation | Academia | Xinyi Huang Independent Researcher EMAIL Kangfei Zhao Beijing Institute of Technology EMAIL Long-Kai Huang Hong Kong Baptist University EMAIL |
| Pseudocode | Yes | Summary of the Procedure. For model editing tasks with multiple available samples, the first stage of Refine Vi T identifies the attention heads most responsible for prediction failures through the following systematic process: 1. Score Calculation: For each attention head, compute all four utility scores (U l,h A , U l,h B , U l,h C , U l,h D ) as permitted by the available data. 2. Candidate List Generation: For each utility, rank the attention heads in descending order and select the top T heads to form a candidate list. 3. Ablation Candidate Generation: For each candidate list, iteratively select the top t heads (for t = 1 to T) as ablation candidates, denoted as St. 4. Validation Utility Evaluation: For each ablation set St, evaluate the validation utility (Eq. 18) on all available samples. The set St with the highest validation utility is chosen as the final set of attention heads for ablation, i.e., Sablate. |
| Open Source Code | Yes | Our code is available at https: //github.com/shanghxy/Model-editing-for-vision-transformers. |
| Open Datasets | Yes | We evaluate the proposed method, Refine Vi T, on the Vi T editing benchmark from [32], the Binary Waterbirds dataset [25], Celeb A [19], Image Net-R [13], and Image Net-A [14]. This dataset combines thousands of waterbird and landbird images from the CUB dataset [29] with water or land backgrounds from the Places dataset [36]. |
| Dataset Splits | Yes | Other samples exhibiting the same type of prediction error as the reference sample are treated as neighboring samples within the editing scope for the generalization rate (GR) computation. Furthermore, 2,071 carefully curated images near the decision boundary of Vi T-B/16, sourced from the validation sets of Image Net-1K [3], Image Net-R [13], and Image Net-Sketch [30], are used as unrelated samples outside the editing scope for the calculation of the locality rate (LR). We randomly select 10 samples per class waterbirds and landbirds for the Waterbirds dataset [25], and young and old celebrities for the Celeb A dataset [19] including both correctly and incorrectly predicted instances from CLIP-Vi T. All experiments are repeated with three random seeds to reduce variance due to random initialization. |
| Hardware Specification | Yes | This allows updates in under 0.3 seconds for 50 epochs on a single NVIDIA A100 (40GB) as the output of the backbone can be cached and reused. |
| Software Dependencies | No | The paper mentions using the 'Adam optimizer' but does not specify any software libraries or their version numbers. For example, it does not state which version of PyTorch or TensorFlow was used. |
| Experiment Setup | Yes | For Refine Vi T, we fix the number of training epochs to 50 and use the Adam optimizer with a learning rate of 0.00002. All experiments use the Adam optimizer with a fixed learning rate of 2 10 5. We set both hyperparameters α and β to 1000. The weight of the cross-entropy loss is fixed at 1, while the weights of Lsuccess(θ) and Llocality(θ) are donated as α and β, respectively. |