Robust Weight Signatures: Gaining Robustness as Easy as Patching Weights?
Authors: Ruisi Cai, Zhenyu Zhang, Zhangyang Wang
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | This paper empirically suggests a surprisingly simple answer: linearly by straightforward model weight arithmetic! We experimentally verify our proposed framework to be remarkably (1) lightweight. since RWSs concentrate on the shallowest few layers and we further show they can be painlessly quantized, storing an RWS is up to 13 more compact than storing the full weight copy; (2) in-situ adjustable. RWSs can be appended as needed and later taken off to restore the intact clean model. We further demonstrate one can linearly re-scale the RWS to control the patched robustness strength; (3) composable. Multiple RWSs can be added simultaneously to patch more comprehensive robustness at once; and (4) transferable. |
| Researcher Affiliation | Academia | Ruisi Cai 1 Zhenyu Zhang 1 Zhangyang Wang 1 1VITA Group, University of Texas at Austin |
| Pseudocode | No | The paper describes its method in prose and equations (e.g., Equation 1, Equation 2) but does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | https://github.com/VITA-Group/Robust_Weight_Signatures |
| Open Datasets | Yes | We use three datasets, CIFAR-10, CIFAR-100 (Krizhevsky et al., 2009) and Tiny-Image Net (mnmoustafa, 2017), with two model architectures, VGG-16 (Simonyan & Zisserman, 2014) and Res Net-50 (He et al., 2016). |
| Dataset Splits | No | The paper does not explicitly specify the training, validation, or test dataset splits (e.g., percentages, sample counts, or k-fold cross-validation setup) used for reproducibility. It mentions using standard datasets and evaluating on corrupted data but no specific splits. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running the experiments, such as GPU models, CPU types, or memory specifications. |
| Software Dependencies | No | The paper mentions "Pytorch" in a footnote (1https://pytorch.org/vision/stable/models.html) but does not provide a specific version number. No other software or libraries with version numbers are specified. |
| Experiment Setup | Yes | All VGG-16 models use a learning rate of 0.01, while all Res Net-50 models use 0.001. We follow the corruption types in (Hendrycks & Dietterich, 2019b) and the corruption severity levels are set to be 5 (strongest) for all experiments by default. |