Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
General-Reasoner: Advancing LLM Reasoning Across All Domains
Authors: Xueguang Ma, Qian Liu, Dongfu Jiang, Ge Zhang, Zejun MA, Wenhu Chen
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We train a series of models and evaluate them on a wide range of datasets covering wide domains like physics, chemistry, finance, electronics etc. Our comprehensive evaluation across these 12 benchmarks (e.g. MMLU-Pro, GPQA, Super GPQA, Theorem QA, BBEH and MATH AMC) demonstrates that General-Reasoner outperforms existing baseline methods, achieving robust and generalizable reasoning performance while maintaining superior effectiveness in mathematical reasoning tasks. |
| Researcher Affiliation | Collaboration | Xueguang Ma, Qian Liu, Dongfu Jiang, Ge Zhang, Zejun Ma, Wenhu Chen University of Waterloo, Vector Institute, Tik Tok, Singapore, M-A-P EMAIL,EMAIL |
| Pseudocode | No | The paper describes mathematical formulations and processes but does not include any clearly labeled pseudocode or algorithm blocks. For example, it presents equations for JGRPO(θ) and ri,t(θ) and describes the generative verifier's inference process, but these are not formatted as pseudocode. |
| Open Source Code | No | We will release the code, data, and model checkpoints based on acceptance. |
| Open Datasets | Yes | We conduct comprehensive evaluations across 12 challenging reasoning benchmarks beyond mathematics, including MMLU-Pro [8], GPQA [10], Super GPQA [11], Theorem QA [12], and BBEH [13], as well as standard mathematical reasoning benchmarks such as MATH-500 [14], GSM8K [15], and Olympiad [16]. |
| Dataset Splits | Yes | GPQA [10]: Graduate-level question answering designed to be resistant to shallow patternmatching or memorization. We use the diamond split in GPQA. |
| Hardware Specification | Yes | Training is conducted on 4 nodes with 8 H100 GPUs per node for up to 700 steps for Qwen2.5 series models. For model initialzied with Qwen3-Base, we train up to 400 steps. Please see Appendix A.5 for detailed hyperparameters of each model checkpoint. |
| Software Dependencies | Yes | We provide the detailed hyperparameters for training our General-Reasoner variants in Table 11. The difference in batch size configurations between the Qwen2.5 and Qwen3 series is due to the limitations in earlier versions of v LLM (e.g., v0.6.3), which did not support engine sleep and wake mechanisms for properly loading and unloading the verifier model. With the newer v LLM version (v0.8.5), we are able to share GPU resources more efficiently by leveraging proper parameter loading and unloading. |
| Experiment Setup | Yes | Training is conducted on 4 nodes with 8 H100 GPUs per node for up to 700 steps for Qwen2.5 series models. For model initialzied with Qwen3-Base, we train up to 400 steps. Please see Appendix A.5 for detailed hyperparameters of each model checkpoint. ... We provide the detailed hyperparameters for training our General-Reasoner variants in Table 11. ... learning_rate 5e-7 ... ppo_mini_batch_size 192 ... kl_loss_coef 0.0001 |