Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
LongMagpie: A Self-synthesis Method for Generating Large-scale Long-context Instructions
Authors: Chaochen Gao, Xing W, Zijia Lin, Debing Zhang, Songlin Hu
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on HELMET, RULER, and Longbench v2 demonstrate that Long Magpie achieves leading performance on long-context tasks while maintaining competitive performance on short-context tasks, establishing it as a simple and effective approach for open, diverse, and scalable long-context instruction data synthesis. |
| Researcher Affiliation | Collaboration | Chaochen Gao1,2, Xing Wu1,2 , Zijia Lin4, Debing Zhang3, Songlin Hu1,2 1Institute of Information Engineering, Chinese Academy of Sciences 2School of Cyber Security, University of Chinese Academy of Sciences 3Xiaohongshu Inc, 4Tsinghua University EMAIL EMAIL, EMAIL |
| Pseudocode | Yes | Algorithm 1 Hybrid SFT Data Construction with short-context Pre-pending and Probabilistic Mixing |
| Open Source Code | No | We will release our data and model in the future version. |
| Open Datasets | Yes | Dataset Generation Using the Long Magpie pipeline described in Section 1, we generate a longcontext instruction dataset using Qwen2.5-70B-Instruct, with documents sampled from Fine Web-Edu [34]. |
| Dataset Splits | No | The paper describes using Fine Web-Edu for document sampling to generate a dataset for training but does not specify how this generated dataset, or any other training dataset, is split into training, validation, or test sets for their experiments. |
| Hardware Specification | Yes | GPU-type H100 GPU-numbers 8 |
| Software Dependencies | No | The paper mentions using Flash Attention-2 and ZeRO but does not provide specific version numbers for these or any other software components used for replication. |
| Experiment Setup | Yes | Table 8: Model Training Configuration. training setting Initial Model Llama-3-8B-NExt Long-512K-Base rotary-emb-base 128,000,000 β1 0.9 β2 0.95 lr 2e 5 precision bfloat16 gradient-clipping 1.0 weight-decay 0.1 lr-decay-style cosine train-iters 250 seq-length 65536 |