Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM
Authors: Bowen Dong, Minheng Ni, Zitong Huang, Guanglei Yang, Wangmeng Zuo, Lei Zhang
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct extensive experiments on MIRAGE, leading to several key insights. |
| Researcher Affiliation | Academia | 1Harbin Institute of Technology 2The Hong Kong Polytechnic University EMAIL EMAIL EMAIL EMAIL |
| Pseudocode | No | The paper describes methods like Curriculum Reinforcement Fine-Tuning and Collaborative Hint Inference in text and mathematical formulas (Section 5) but does not provide structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Answer: [Yes] Justification: https://github.com/Dong Sky/MIRAGE |
| Open Datasets | Yes | We propose MIRAGE, the first benchmark for evaluating multimodal reasoning hallucinations in MLLMs. |
| Dataset Splits | Yes | To address these challenges, we propose MIRAGE, a diagnostic benchmark specifically designed to isolate reasoning-induced hallucinations in MLLMs. As shown in Fig. 1, MIRAGE contains 1,329 questions where MLLMs demonstrate accurate visual perception but exhibit defective reasoning. |
| Hardware Specification | Yes | All the experiments are conducted on 8 NVIDIA RTX A6000 GPUs. |
| Software Dependencies | No | All programs are constructed by Py Torch [48] toolkit and v LLM [29] framework. |
| Experiment Setup | Yes | The batch size is 128. For each training sample, the rollout samples G is 8 by default. The initial learning rate is 1e-6, both warmup strategy and cosine learning rate scheduler are adopted to stabilize training. We optimize Logos by 10 epochs using Adam W [40] during each stage. The number of CRFT stages is set to 1, and we will discuss this choice in Sec. 6.2. |