Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Knowledge Starts with Practice: Knowledge-Aware Exercise Generative Recommendation with Adaptive Multi-Agent Cooperation
Authors: Yangtao Zhou, Hua Chu, chen, Ziwen Wang, Jiacheng Liu, Jianan Li, Yueying Feng, Xiangming Li, Zihan Han, Qingshan Li
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To evaluate Exe Gen, we design a comprehensive evaluation protocol that integrates GPTbased scoring, statistical analysis, and human evaluations from both students and teachers. Extensive experiments on real-world educational datasets and a practical deployment in college education demonstrate the effectiveness and superiority of Exe Gen. |
| Researcher Affiliation | Academia | 1Xidian University, 2Zhejiang University EMAIL {lijianan}@xidian.edu.cn EMAIL {yueyingf}@zju.edu.cn |
| Pseudocode | No | The paper describes the 'Adaptive Multi-Agent Cooperation Framework' and 'Human-simulated Knowledge Perception Mechanism' in detail, outlining the processes and interactions. However, it presents these as textual descriptions and diagrams (Figure 2, Figure 3) rather than formal pseudocode blocks or algorithms with numbered steps formatted as code. |
| Open Source Code | Yes | The code is available at https://github.com/dsz532/exe Gen. |
| Open Datasets | Yes | We evaluate Exe Gen on the MOOCCube X dataset [55] with a KG, and the detailed dataset statistics are provided in Appendix A.1. |
| Dataset Splits | No | The paper mentions extracting "all student interaction data from the computer science and technology domain" from the MOOCCube X dataset and conducting human evaluations with "20 college student volunteers" and "20 randomly sampled student groups" for teachers. However, it does not specify explicit training, validation, or test splits (e.g., percentages or counts) for the primary dataset used to develop and evaluate the models quantitatively, beyond the context of the human evaluations. |
| Hardware Specification | Yes | All experiments were conducted on a Linux machine configured with two 4090 GPUs. |
| Software Dependencies | Yes | We implemented our proposed Exe Gen using non-distributed training in Python 3.8.19 and Py Torch 2.3.0. |
| Experiment Setup | Yes | We selected GPT-4o as our base LLM and used the Open AI API, without fine-tuning applied. In our experiments, generating knowledge-based exercise recommendations for each student using the GPT-4o API incurs an average financial cost of 0.1787 dollars. The configurable parameter m controls the number of retrieved triplets, affecting the depth and breadth of relevant knowledge. For exercise quantity, the default setting is 10, but can be adjusted to match the learning goals of students. This process continues iteratively until the exercises satisfy all quality standards or the iterative process reaches a maximum of 10 rounds. |