Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Enhancing Vision-Language Model Reliability with Uncertainty-Guided Dropout Decoding
Authors: Yixiong Fang, Ziran Yang, Zhaorun Chen, Zhuokai Zhao, Jiawei Zhou
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Evaluations on benchmarks including CHAIR, THRONE, and MMBench demonstrate that DROPOUT DECODING significantly reduces object hallucinations (OH) and enhances both reliability and quality of LVLM outputs across diverse visual contexts. |
| Researcher Affiliation | Academia | Yixiong Fang Carnegie Mellon University EMAIL Ziran Yang Princeton University EMAIL Zhaorun Chen University of Chicago EMAIL Zhuokai Zhao University of Chicago EMAIL Jiawei Zhou Stony Brook University EMAIL |
| Pseudocode | Yes | Algorithm 1 Pseudocode of DROPOUT DECODING. |
| Open Source Code | Yes | Code is released at https://github.com/kigb/Dropout Decoding. |
| Open Datasets | Yes | Evaluations on benchmarks including CHAIR, THRONE, and MMBench demonstrate that DROPOUT DECODING significantly reduces object hallucinations (OH) and enhances both reliability and quality of LVLM outputs across diverse visual contexts. [...] For OH, we use the CHAIR [33] and THRONE [34] metrics to assess the performance of different decoding methods on the MSCOCO dataset. Additionally, we employ MMBench [35] to evaluate the overall generation quality and general ability of these methods. |
| Dataset Splits | Yes | Our experiment is conducted on the MSCOCO 2014 test set, where we randomly sample 500 images across 3 random seeds. The average and standard deviation across these seeds are reported in our result table. |
| Hardware Specification | Yes | Using v LLM and LLa VA-1.5 on 4 A800 80GB GPUs, GPU memory usage was 38.12 GB with efficient KV caching, unchanged between greedy decoding and our method without preliminary passes. |
| Software Dependencies | No | The paper mentions Huggingface Transformers but does not provide specific version numbers for any software dependencies. |
| Experiment Setup | Yes | The experimental setup of DROPOUT DECODING is shown in Table 6. We set the maximum new tokens to 512 to ensure the complete generation of models, therefore achieving more reliable results from CHAIR and THRONE. In MMBench, as all questions are single-choice questions, we set the maximum new tokens to 1 for a more precise evaluation. We set other parameters in generation to greedy for more stable and repeatable results. Parameters CHAIR THRONE MMBench Max New Tokens 512 512 256 Top-k False Top-p 1 Temperature τ 1 Number Beams 1 Table 6: Parameter settings used in our experiments. Moreover, we provide the hyperparameter settings of our baselines. OPERA s hyperparameters can be referred to Table 7; VCD s hyperparameters can be referred to Table 8. |