Bayesian-guided Label Mapping for Visual Reprogramming
Authors: Chengyi Cai, Zesheng Ye, Lei Feng, Jianzhong Qi, Feng Liu
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments conducted on both pretrained vision models (e.g., Res Ne Xt) and vision-language models (e.g., CLIP) demonstrate the superior performance of BLM over existing label mapping methods. The success of BLM also offers a probabilistic lens through which to understand and analyze the effectiveness of VR. Our code is available at https://github.com/tmlr-group/Bayesian LM. |
| Researcher Affiliation | Academia | 1The University of Melbourne 2Singapore University of Technology and Design {chengyi.cai1,zesheng.ye,jianzhong.qi}@unimelb.edu.au feng_lei@sutd.edu.sg fengliu.ml@gmail.com |
| Pseudocode | Yes | The learning pipeline of BLM is shown in Algorithm 1, while that of BLM+ is shown in Algorithm 2. The completed pseudocode for all LM methods (RLM, FLM, ILM, BLM, BLM+) and a more detailed discussion of involved matrix operations are in Appendix D. |
| Open Source Code | Yes | Our code is available at https://github.com/tmlr-group/Bayesian LM. |
| Open Datasets | Yes | To show the effectiveness of BLM, experiments are conducted on 12 widely used datasets, with BLM and BLM+ being applied to different input VR methods padding and watermarking on pretrained Res Net and Res Ne Xt (see Section 5). The ablation study and parameter analysis are also included, along with visualization results and discussions of why VR is effective. BLM and BLM+ are also applied to vision-language models (see Appendix L) to demonstrate their general applicability. Detailed dataset information is presented in Table 4. |
| Dataset Splits | Yes | Additional experiments are conducted using a validation set and training set split of 30% and 70% to find optimal hyper-parameters for each dataset. |
| Hardware Specification | Yes | All experiments are conducted on a single A100 GPU and the average accuracy of three distinct random seeds are reported. |
| Software Dependencies | No | For training input VR patterns, we apply the Adam optimizer [26] with an initial learning rate of 0.01. The number of epochs is 200, with the learning rate decay being 0.1, scheduled at epochs 100 and 145. All experiments are conducted on a single A100 GPU and the average accuracy of three distinct random seeds are reported. |
| Experiment Setup | Yes | For training input VR patterns, we apply the Adam optimizer [26] with an initial learning rate of 0.01. The number of epochs is 200, with the learning rate decay being 0.1, scheduled at epochs 100 and 145. |