Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Reasoning Planning for Language Models
Authors: Ngoc Bao Nguyen, Trung Hieu Nguyen, Ruifeng She, Xiaojin Fu, Viet Anh Nguyen
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on diverse mathematical reasoning tasks show that EPIC consistently selects optimal reasoning methods, improving accuracy while reducing computational overhead. Our code can be found at https://github.com/nguyenngocbaocmt02/EPIC. Extensive experiments on the MATH dataset demonstrate EPIC s advantage: compared to individual reasoning models in the universe of methods, EPIC can reduce the number of tokens (or cost) by 75% while maintaining the same level of accuracy. Section 5 Numerical Experiments. |
| Researcher Affiliation | Collaboration | 1 The Chinese University of Hong Kong 2 Huawei Noah s Ark Lab |
| Pseudocode | No | The paper describes the EPIC framework and its components, including contrastive loss and regularization terms, using mathematical formulations and descriptive text. However, it does not present any structured pseudocode blocks or algorithms labeled as such for the overall method. |
| Open Source Code | Yes | Our code can be found at https://github.com/nguyenngocbaocmt02/EPIC. |
| Open Datasets | Yes | We use the MATH dataset [Hendrycks et al., 2021] as a training set, utilizing its training split of 7,500 math problems with solutions, as defined in Hendrycks et al. [2021]. For the code generation experiment, we use the Live Code Bench dataset [Jain et al., 2025]. |
| Dataset Splits | Yes | We use the MATH dataset [Hendrycks et al., 2021] as a training set, utilizing its training split of 7,500 math problems with solutions, as defined in Hendrycks et al. [2021]. For evaluation, we test on the MATH500 test split, which contains 500 samples, as defined in Lightman et al. [2023b]. |
| Hardware Specification | Yes | All experiments are conducted on a single machine with 8 NVIDIA RTX A5000 GPU and Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz. |
| Software Dependencies | No | The paper mentions specific language models and reward models used (e.g., Qwen2.5-Math-7B-Instruct, math-shepherd-mistral-7b-prm) and a sentence embedding model (all-Mini LM-L6-v2) along with their HuggingFace links. However, it does not explicitly state versions for general software dependencies like Python, PyTorch, or CUDA. |
| Experiment Setup | Yes | To measure average token counts, we set the hyperparameter max new token to 2048 for all methods and compute the average number of tokens generated. We apply the regularization parameter τ = 10 3 based on its better numerical results than other values shown in Appendix D. Our method with λ = 0.25 achieves an accuracy of 86.4%. ... we vary the temperature of decoding in the set {0.4, 0.7, 1.0} to explore different levels of diversity. |