Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Understanding and Rectifying Safety Perception Distortion in VLMs
Authors: Xiaohan Zou, Jian Kang, George Kesidis, Lu Lin
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments demonstrate that Shift DC significantly enhances safety alignment without impairing model utility. The code is available at https://github.com/Renovamen/Shift DC. ... Through experiments on three VLM safety benchmarks, three visual reasoning utility benchmarks, and five different VLMs, we demonstrate that Shift DC significantly enhances the alignment ability of VLMs without compromising their general performance. |
| Researcher Affiliation | Academia | 1The Pennsylvania State University 2MBZUAI 3University of Rochester EMAIL, EMAIL, EMAIL |
| Pseudocode | No | The paper describes the Shift DC method textually and through mathematical equations (Eq. 5, 6, 7) and a conceptual figure (Figure 3), but it does not include a distinct block explicitly labeled 'Pseudocode' or 'Algorithm'. |
| Open Source Code | Yes | The code is available at https://github.com/Renovamen/Shift DC. |
| Open Datasets | Yes | We evaluate the safety of VLMs responses using three benchmarks: MM-Safety Bench [5], Fig Step [7], and Jail Break V-28K [48]. ... Experiments are conducted on popular VLM utility benchmarks, MME [49], MM-Vet [50] and MMBench [51]... Dunsafe vl and Dunsafe tt are constructed from MM-Safety Bench [5], while Dsafe vl and Dsafe tt are sourced from LLa VA-Instruct-80k [3]. |
| Dataset Splits | Yes | Both Dtt and Dvl use a 4:1 split for training and testing. ... For linear probing as described in Section 4, 128 samples are used for training, and the remaining 32 for testing. |
| Hardware Specification | No | The paper discusses inference efficiency and execution times (Table 13, 14) and evaluates models like LLa VA-1.5-7B, Mini GPT-4-7B, etc. but does not explicitly provide details about the specific hardware (e.g., GPU models, CPU types, memory) used to run the experiments. |
| Software Dependencies | No | The paper does not explicitly mention specific software dependencies with version numbers (e.g., Python, PyTorch, CUDA versions) used for implementation or experimentation. |
| Experiment Setup | Yes | In our implementation, the maximum token length is set to 1024. ... In our preliminary experiments on understanding the mechanism behind safety perception distortion, Dunsafe vl and Dunsafe tt are constructed from MM-Safety Bench [5], while Dsafe vl and Dsafe tt are sourced from LLa VA-Instruct-80k [3]. ... The input template used to prompt the VLM for caption generation is shown in Table D.1. ... The template used for this classification is shown in Table D.2, prompting the VLM to respond with yes or no. |