Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
GAM-Agent: Game-Theoretic and Uncertainty-Aware Collaboration for Complex Visual Reasoning
Authors: jusheng zhang, Yijia Fan, Wenjun Lin, Ruiqi Chen, Haoyi Jiang, Wenhao Chai, Jian Wang, Keze Wang
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments on four challenging benchmarks MMMU, MMBench, MVBench, and V*Bench demonstrate that GAM-Agent significantly improves performance across various VLM backbones. Notably, GAM-Agent boosts the accuracy of small-to-mid scale models (e.g., Qwen2.5-VL-7B, Intern VL3-14B) by 5 6%, and still enhances strong models like GPT-4o by up to 2 3%. Our approach is modular, scalable, and generalizable, offering a path toward reliable and explainable multi-agent multimodal reasoning. |
| Researcher Affiliation | Collaboration | Jusheng Zhang1,*, Yijia Fan1,*, Wenjun Lin1, Ruiqi Chen1, Haoyi Jiang1, Wenhao Chai2, Jian Wang3, Keze Wang1, 1Sun Yat-sen University 2Princeton University 3Snap Inc. Corresponding author: EMAIL |
| Pseudocode | No | The paper describes the methodology using textual explanations and mathematical formulas in Section 2 and Appendix D, such as defining GAM-Agent as a six-tuple S = (E, A, Φ, M, P, D) and detailing its components, but it does not contain a clearly labeled pseudocode block or algorithm. |
| Open Source Code | No | Open access to data and code Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [No] Justification: To preserve anonymity at submission time, the code repository is not made public; however, the paper and supplementary provide detailed descriptions of the method and experimental settings to support faithful reproduction. |
| Open Datasets | Yes | Experiments on four challenging benchmarks MMMU, MMBench, MVBench, and V*Bench demonstrate that GAM-Agent significantly improves performance across various VLM backbones. Notably, GAM-Agent boosts the accuracy of small-to-mid scale models (e.g., Qwen2.5-VL-7B, Intern VL3-14B) by 5 6%, and still enhances strong models like GPT-4o by up to 2 3%. |
| Dataset Splits | Yes | The evaluation is conducted on three challenging benchmarks: MMMU[64] (multi-discipline multimodal understanding), MMBench_V11_Test (visual reasoning and perception), and MVBench_Test[34] (video temporal reasoning). |
| Hardware Specification | Yes | Open-source models (Qwen2.5VL, Intern VL3) were deployed locally on NVIDIA A100 GPUs, while the closed-source GPT-4o-0513 was accessed via the Open Router API. Detailed hyperparameter settings and other experimental configurations are provided in the Supplementary Material. On each benchmark, our GAM-Agent achieves higher overall accuracy (Overall ACC, %) than its corresponding base model, with improvements summarized in Table 1. Unless otherwise stated, all experiments in this paper are performed on NVIDIA A100 GPU. |
| Software Dependencies | No | The paper mentions using open-source models like Qwen2.5VL and Intern VL3 and accessing GPT-4o-0513 via API, but does not specify explicit version numbers for software dependencies such as Python, PyTorch, or CUDA. |
| Experiment Setup | Yes | For all experimental runs involving debate frameworks, we set a maximum of 3 debate rounds (max_debate_round=3). Each framework utilized 3 expert agents (N = 3) for generating initial responses and argumentation, and 3 critic agents (Ncrit = 3) for frameworks requiring critique/verification, such as GAM-Agent. We used greedy decoding for text generation. Open-source models (Qwen2.5VL, Intern VL3) were deployed locally on NVIDIA A100 GPUs, while the closed-source GPT-4o-0513 was accessed via the Open Router API. Detailed hyperparameter settings and other experimental configurations are provided in the Supplementary Material. |