Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
MemEIC: A Step Toward Continual and Compositional Knowledge Editing
Authors: Jin Seong, Jiyun Park, Wencke Liermann, Hongseok Choi, Yoonji Nam, Hyun Jun Kim, Soojong Lim, Namhoon Lee
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments demonstrate that Mem EIC significantly improves performance on complex multimodal questions and effectively preserves prior edits, setting a new benchmark for CCKE in LVLMs. |
| Researcher Affiliation | Academia | Electronics and Telecommunications Research Institute, Republic of Korea1 POSTECH2, Sungkyunkwan University, Korea3 |
| Pseudocode | No | The methodology section (Section 3) describes the proposed framework in narrative text without explicit pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our project is available at https://github.com/MemEIC/MemEIC. |
| Open Datasets | Yes | Our new benchmark CCKEB, an extended version of VLKEB [11] originally focused on visual editing by enabling compositional edits that involve both visual and textual modalities. CCKEB augments 5,000 visual editing instances from the VLKEB training split with their corresponding textual edits, thereby forming a fully compositional benchmark. Concretely, each image instance in CCKEB is paired with two coordinated edits: (1) a visual edit that updates the image’s identity label, and (2) a textual edit that revises a factual statement about the (new) entity depicted in the image. ... we leverage the multi-modal knowledge graph MMKG [41]. ... We extract candidate triples for the training set using the DB15K [48] knowledge base... extract the set of relations associated with each object using the FB15K [50] and DB15K [48] knowledge graphs, in that order. |
| Dataset Splits | Yes | CCKEB augments 5,000 visual editing instances from the VLKEB training split with their corresponding textual edits, resulting in 5,000 visual-textual training pairs. The same procedure is applied to the evaluation split, resulting in 1,278 compositional evaluation pairs. |
| Hardware Specification | Yes | All experiments are performed on an NVIDIA A100 80GB GPU utilizing the PyTorch framework. |
| Software Dependencies | No | All experiments are performed on an NVIDIA A100 80GB GPU utilizing the PyTorch framework. The parameters used in the experiments are summarized in Table 8, and detailed configuration files are available in the code repository. For detailed parameters and configurations specific to Mem EIC, please refer to Section D.3. |
| Experiment Setup | Yes | The parameters used in the experiments are summarized in Table 8, and detailed configuration files are available in the code repository. For detailed parameters and configurations specific to Mem EIC, please refer to Section D.3. (Table 8 lists 'Optimizer' and 'LR' for various models and methods). |