Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
MoORE: SVD-based Model MoE-ization for Conflict- and Oblivion-Resistant Multi-Task Adaptation
Authors: Shen Yuan, Yin Zheng, Taifeng Wang, binbin liu, Hongteng Xu
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on various datasets demonstrate that Mo ORE outperforms existing multi-task adaptation methods consistently, showing its superiority in terms of conflict and oblivion-resistance. |
| Researcher Affiliation | Collaboration | 1Gaoling School of Artificial Intelligence, Renmin University of China 2Byte Dance |
| Pseudocode | No | The paper describes the method using mathematical equations and textual descriptions, but does not include a dedicated pseudocode block or algorithm listing. |
| Open Source Code | Yes | The code is available at https://github. com/Da Shen Zi721/Mo ORE. |
| Open Datasets | Yes | The CSR-MTL dataset is constructed by nine tasks, including ARC-Challenge (ARC-C), ARC-Easy (ARC-E) [9], Open Book QA (OBQA) [45], PIQA [5], Social IQA (SIQA) [54], Bool Q [8], Hellaswag (Hella S) [77], Winogrande (Wino G) [53], and Commonsense QA (CSQA) [59]. The NLU-MTL dataset consists of seven tasks from GLUE [63], including Co LA, SST-2, MRPC, QQP, MNLI, QNLI, and RTE. The dataset includes seven tasks, including MMLU [24, 23], IFEval [83], BIG-Bench Hard (BBH) [58], GPQA [51], Human Eval [7], MBPP [4], and GSM-8K [10]. |
| Dataset Splits | Yes | Detailed information about the CSR-MTL, the NLU-MTL and the OR-MTL is presented in Tables 8, Table 9 and Table 10, respectively. These tables include the sizes of the training and test sets, as well as the task types. |
| Hardware Specification | Yes | Both training and testing are conducted on one NVIDIA A100 GPU. |
| Software Dependencies | No | The paper does not explicitly list software dependencies with version numbers. It mentions Adam W [39] as the optimizer, but no specific software environment details are provided. |
| Experiment Setup | Yes | The detailed hyperparameter setups are presented in Table 7. Both training and testing are conducted on one NVIDIA A100 GPU. Table 7: Hyperparameter configurations of Mo ORA on the CSR-MTL and the NLU-MTL. Hyperparameters: Cutoff Length 512, Batch Size 8 / 64, Epochs 2 / 5, Learning Rate 3E-04, LR scheduler Warmup-Stable-Decay, Warmup Ratio 5%, Decay Ratio 5%, Optimizer Adam W, Dropout Rate 0.0, Target Modules Q, K, V, O, Up, Down, Gate, Dt 128, Ds 64, L 8. |