Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
AdvEDM: Fine-grained Adversarial Attack against VLM-based Embodied Agents
Authors: Yichen Wang, Hangtao Zhang, Hewen Pan, Ziqi Zhou, Xianlong Wang, Peijin Guo, Lulu Xue, Shengshan Hu, Minghui Li, Leo Yu Zhang
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conducted experiments in both general evaluation scenarios and EDM tasks. The general scenario involves image description, which is consistent with existing adversarial attack evaluation scenarios in general VLMs. The EDM tasks include autonomous driving and robotic arm manipulation. Besides, more visualization results are provided in Appendix B and our webpage. |
| Researcher Affiliation | Academia | 1National Engineering Research Center for Big Data Technology and System 2Services Computing Technology and System Lab 3Cluster and Grid Computing Lab 4Hubei Engineering Research Center on Big Data Security 5Hubei Key Laboratory of Distributed System Security 6School of Cyber Science and Engineering, Huazhong University of Science and Technology 7School of Computer Science and Technology, Huazhong University of Science and Technology 8Department of Computer Science, City University of Hong Kong 9School of Software Engineering, Huazhong University of Science and Technology 10 School of Information and Communication Technology, Griffith University EMAIL EMAIL, EMAIL |
| Pseudocode | No | The paper describes the methodologies ADVEDM-R and ADVEDM-A using mathematical equations (Eq. 3-12) and a pipeline diagram (Fig. 3), but it does not include a clearly labeled pseudocode or algorithm block. |
| Open Source Code | Yes | More demos of our attacks in real-world scenarios can be found on our website https://advedm.github.io/. Part of our source code is in the supplementary material. The complete code will be released after publication. |
| Open Datasets | Yes | Datasets. For general scenarios, we select MS-COCO 2014 [57, 58]. For the autonomous driving scenario, we choose Dolphins Benchmark [21] and Drive LM-nu Scenes [4] that are specialized for this task, while for the robotic arm manipulation task, we sample 100 images from the physical world and construct instructions and actions. |
| Dataset Splits | No | For general scenarios, we select MS-COCO 2014 [57, 58]. For the attack target, we randomly select an object in the image to remove its semantics or choose an object not in the image to inject its semantics. We randomly select 1000 images to generate adversarial examples and record the average ASR, SPR and SS. Settings. We conduct experiments on two specialized datasets, Dolphins Benchmark and Drive LMnu Scenes, with three general VLMs (LLAVA-v2, Mni GPT-4, and Otter-Image) and Dolphins (Dol) [21], a VLM designed for decision-making task in autonomous driving. For each dataset, we randomly select 100 road scene images and choose common objects in the road as target, such as vehicles, pedestrians, and traffic lights. Settings. We take the robotic arm manipulation task as an example, where various objects are placed on desktop, and then 100 images are captured to form the dataset. |
| Hardware Specification | Yes | Experimental environment. All experiments are conducted on NVIDIA A100-SXM4 GPUs, each equipped with 80GB of memory. |
| Software Dependencies | No | The paper mentions the use of 'Adam optimizer [59]' and 'GPT-3.5-turbo' for LLM-as-judge. However, it does not specify version numbers for programming languages, libraries, or other software components used for implementing the methods described. |
| Experiment Setup | Yes | Parameter Settings of our methods. For ADVEDM-R, we select the top 20% of image patches with the highest similarity to the target object s text embedding and mask their pixels to generate masked image M. We set w1, w2 and w3 in Eq. (3) to 0.5, 2, and 0.2, respectively. The optimization is performed for 500 iterations using the Adam optimizer [59] with a learning rate of 0.005. For ADVEDM-A, we We select a 100 100 pixel region in the image for semantic injection. When reallocating attention weights, we set the scaling factor β = 0.4 in Eq. (12), and the fusion weight α of [CLS] tokens is 0.5 in Eq. (11). During the optimization process, w1 to w3 is set to 0.8, 2, and 0.3, respectively. The remaining optimization settings are kept identical to those of ADVEDM-R. |