MELO: Enhancing Model Editing with Neuron-Indexed Dynamic LoRA
Authors: Lang Yu, Qin Chen, Jie Zhou, Liang He
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results show that our proposed MELO achieves state-of-the-art editing performance on three sequential editing tasks (document classification, question answering and hallucination correction), while requires the least trainable parameters and computational cost. |
| Researcher Affiliation | Academia | 1School of Computer Science and Technology, East China Normal University 2Shanghai Institute of AI for Education, East China Normal University |
| Pseudocode | No | The paper describes the MELO framework and its components (Dynamic LoRA, Vector Database) through text and diagrams, but it does not provide any structured pseudocode blocks or algorithms. |
| Open Source Code | Yes | Code is available at https://github.com/Bruth YU/MELO |
| Open Datasets | Yes | SCOTUS is a subset of Fairlex (Chalkidis et al. 2022); zs RE is a question answering (QA) dataset built upon zero-shot relation extraction (Levy et al. 2017); Hallucination is introduced by (Manakul, Liusie, and Gales 2023) |
| Dataset Splits | No | The paper mentions splitting data into 'edits' and 'holdouts' for some datasets and using 'upstream datasets' for locality evaluation. However, it does not provide specific percentages, sample counts, or explicitly refer to a 'validation set' with sufficient detail to reproduce data partitioning for all experiments. |
| Hardware Specification | Yes | With a single Nvidia RTX 3090 GPU, we investigate the editing speed and the amount of extra parameters used on zs RE dataset. |
| Software Dependencies | No | Our proposed MELO is implemented based on the huggingface library PEFT2, which can be easily integrated into multiple LLM backbones for model editing. While PEFT is mentioned, no specific version number is provided for PEFT or any other software dependency, which prevents reproducible software environment setup. |
| Experiment Setup | Yes | Unless otherwise stated, the default hyper-parameter settings of MELO for different backbones are provided in Table 1. Table 1 lists specific values for 'Partial Rank', 'Initial Radius', 'Batch Iteration', and 'Learning Rate' for different LLM backbones (BERT, T5-Small, T5-Large, GPT2-XL). |