Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
FOCUS: Internal MLLM Representations for Efficient Fine-Grained Visual Question Answering
Authors: Liangyu Zhong, Fabio Philipp Rosenthal, Joachim Sicking, Fabian Hüger, Thorsten Bagdonat, Hanno Gottschalk, Leo Schwinn
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate FOCUS on the fine-grained VQA datasets V*Bench [38], HRBench-4K [37] and MME-Real World-Lite [43]. Across the first three datasets, our method achieves on average 42% higher accuracy over the vanilla MLLMs when using LLa VA-1.5 and 17% when using LLa VA-One Vision, while improving LLa VA-One Vision by 6% on the multi-domain MME-Real World-Lite dataset. Moreover, FOCUS achieves comparable or superior performance w.r.t. the state-of-the-art baseline Zoom Eye [31] while being 3.5 4.5 more efficient with LLa VA-1.5 and 3 6.5 more efficient with LLa VA-One Vision. ... We conduct an ablation study that provides insights on how FOCUS leverages MLLM-internal knowledge for efficient visual cropping. |
| Researcher Affiliation | Collaboration | 1Technical University of Berlin, 2Technical University of Munich, 3CARIAD SE, 4Volkswagen AG |
| Pseudocode | Yes | We provide Py Torch-style pseudocode in Fig. 7. ... Algorithm Pseudocode for proposal ranking of ROIs for fine-grained VQA |
| Open Source Code | No | Justification: All datasets and MLLMs are publicly available. We are waiting for internal clearance to release our code upon acceptance. |
| Open Datasets | Yes | We evaluate FOCUS on the fine-grained VQA datasets V*Bench [38], HRBench-4K [37] and MME-Real World-Lite [43]. |
| Dataset Splits | No | We evaluate FOCUS on several fine-grained VQA benchmarks: V*Bench [38], HRBench [37], and MME-Real World-Lite [43]. ... Each dataset contains one question per image. To improve evaluation robustness, HRBench permutes the positions of the answer options, yielding a total of 800 question answer pairs across 200 images. |
| Hardware Specification | Yes | We run all experiments presented in Sec. 4 and App. C on identical hardware, namely compute instances equipped with an NVIDIA A100 80GB GPU, an AMD EPYC 7V13 CPU, and 220 GB of RAM. |
| Software Dependencies | Yes | FOCUS s software environment includes CUDA 12.2, Py Torch 2.6.0, and the Hugging Face transformers library in version 4.46.0. |
| Experiment Setup | Yes | For LLa VA-1.5, we use representations from the 14th to the 32nd layer (l = 14, L = 32), and for LLa VA-One Vision from the 14th to the 28th layer (l = 14, L = 28). To evaluate performance under varying computational budgets, we adjust only the number of steps, setting nsteps {1, 2, 3, 4, 6, 8}. We describe the configuration of the remaining hyperparameters of FOCUS in App. A.3; for an analysis of FOCUS s hyperparameter sensitivity, see Sec. 4.3. ... Table 7: Hyperparameters of FOCUS. |