Sample-specific Masks for Visual Reprogramming-based Prompting
Authors: Chengyi Cai, Zesheng Ye, Lei Feng, Jianzhong Qi, Feng Liu
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To further substantiate the efficacy of SMM, we conduct empirical evaluations spanning 11 widely used datasets, incorporating ablation studies that discern the impact of individual SMM components. This is complemented by analysis and interpretations of the generated masks, alongside a comparative visualization of feature spaces. Notably, we demonstrate the effectiveness of SMM with both pretrained ResNet and ViT (Table 1 and 2), validating that SMM is compatible with commonly used classifier architectures. |
| Researcher Affiliation | Academia | 1School of Computing and Information Systems, The University of Melbourne 2Information Systems Technology and Design Pillar, Singapore University of Technology and Design. |
| Pseudocode | Yes | Algorithm 1 Visual Reprogramming with SMM; Algorithm 2 Computing Frequency Distribution of [f P(fin(xi|θ)), y T]; Algorithm 3 Frequent Label Mapping (f Flm out ); Algorithm 4 Iterative Label Mapping (f Ilm out ) |
| Open Source Code | Yes | Our code is available at https: //github.com/tmlr-group/SMM. |
| Open Datasets | Yes | target tasks include CIFAR10, CIFAR100 (Krizhevsky, 2009), SVHN (Yuval, 2011), GTSRB (Houben et al., 2013), Flowers102 (Nilsback & Zisserman, 2008), DTD (Cimpoi et al., 2014), UCF101 (Soomro et al., 2012), Food101 (Bossard et al., 2014), Euro SAT (Helber et al., 2019), Oxford Pets (Parkhi et al., 2012), SUN397 (Xiao et al., 2010). |
| Dataset Splits | Yes | We follow Chen et al. (2023) to split the datasets. Detailed dataset information is included in Appendix C. The 11 datasets used for the experiments are summarized in Table 6, while the corresponding training parameters are listed in Table 9. |
| Hardware Specification | Yes | Experiments are run with three seeds on a single A100 GPU and the averaged test accuracy is reported. |
| Software Dependencies | No | The paper does not provide specific version numbers for any software dependencies used for the experiments (e.g., Python, PyTorch, CUDA versions). |
| Experiment Setup | Yes | The training details of the mask generator are included in Appendix C. Experiments are run with three seeds on a single A100 GPU and the averaged test accuracy is reported. To ensure that all the methods are fairly compared, in training the shared noise pattern, we apply the same learning rate and milestones following Chen et al. (2023), with 0.01 being the initial learning rate and 0.1 being the learning rate decay. Two hundred epochs are run in total, and the 100th and the 145th epochs are the milestones. Table 9. Detailed Model Training Parameter Settings of Our Mask Generator (where b, α and γ denote batch size, initial learning rate and learning rate decay, respectively) |