Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Self-Verification Provably Prevents Model Collapse in Recursive Synthetic Training
Authors: Shi Fu, Yingjie Wang, Yuzhu Chen, Li Shen, Dacheng Tao
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Appendix K. Additional Experimental Analysis and Future Work. We provide further empirical validation for our assumptions, present experimental results demonstrating how self-verification prevents model collapse, and discuss potential directions for future work. Table 2: Prediction Error Across Recursive Training Rounds for Different Strategies. |
| Researcher Affiliation | Academia | 1College of Computing and Data Science, Nanyang Technological University, Singapore, 2University of Science and Technology of China, Hefei, China, 3Shenzhen Campus of Sun Yat-sen University, Shenzhen, China EMAIL, EMAIL, EMAIL, EMAIL, EMAIL |
| Pseudocode | No | The paper describes methods through mathematical formulations and textual explanations but does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain any explicit statements about releasing source code, nor does it provide links to code repositories or mention code in supplementary materials. |
| Open Datasets | Yes | To further validate this, we conducted experiments on the MATH dataset with Phi3.5-Mini, analyzing the log probabilities of correct versus incorrect responses. |
| Dataset Splits | No | The paper describes data generation for experiments and analysis on the MATH dataset, but it does not specify training, validation, or test dataset splits, percentages, or sample counts. |
| Hardware Specification | No | The paper mentions training a GPT-2 model but does not provide any specific hardware details such as GPU models, CPU types, or other compute resources used for running experiments. |
| Software Dependencies | No | The paper mentions using a GPT-2 model and Phi3.5-Mini but does not specify any software dependencies with version numbers, such as programming languages, libraries, or frameworks. |
| Experiment Setup | Yes | Specifically, we trained a 12 layer, 8 head GPT-2 model (with a hidden size of 256) to recursively perform in context learning of linear functions from the class: F = f | f(x) = w x, w R5. For each prompt, we sampled x1, . . . , xk, xquery and w independently from N(0, Id). The model s task was to predict yquery = w xquery. ... For each training instance, we sampled 20 candidate responses, selected the one with the highest confidence (corresponding to γ = 0), and trained the model exclusively on this verified data. |