Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
RVCL: Evaluating the Robustness of Contrastive Learning via Verification
Authors: Zekai Wang, Weiwei Liu
JMLR 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on various benchmark models and datasets validate theoretical findings, and further demonstrate RVCL s capability to evaluate the robustness of both CL encoders and images. Extensive experiments on benchmark datasets (Mnist, CIFAR-10, CIFAR-100) and diverse models (CNN, Res Net). |
| Researcher Affiliation | Academia | Zekai Wang EMAIL Weiwei Liu EMAIL School of Computer Science Institute of Artificial Intelligence National Engineering Research Center for Multimedia Software Hubei Key Laboratory of Multimedia and Network Communication Engineering Wuhan University Wuhan, Hubei, China |
| Pseudocode | Yes | Algorithm 1 Deterministic verification for CL... Algorithm 2 Probabilistic verification for CL |
| Open Source Code | Yes | Our code is available at https://github.com/wzekai99/RVCL-JMLR. |
| Open Datasets | Yes | Extensive experiments on benchmark datasets (Mnist, CIFAR-10, CIFAR-100)... Mnist (Le Cun and Cortes, 2010) and CIFAR-10 (Krizhevsky and Hinton, 2009). |
| Dataset Splits | Yes | Mnist (Le Cun and Cortes, 2010)... which contains 60,000 training images and 10,000 testing images with 10 classes. CIFAR-10 and CIFAR-100 contain 50,000 training and 10,000 testing images with 10 classes and 100 classes, respectively. |
| Hardware Specification | Yes | Our experiments are conducted on a Ubuntu 64-Bit Linux workstation, having 10-core Intel Xeon Silver CPU (2.20 GHz) and NVIDIA Ge Force RTX 3090 GPU with 24GB graphics memory. |
| Software Dependencies | No | The paper mentions using 'Adam' optimizer and refers to common deep learning models (CNN, ResNet) and libraries (CROWN, CBC), but does not provide specific version numbers for any software dependencies. |
| Experiment Setup | Yes | We set the step size of instance-wise attack α = 0.007, the number of PGD maximize iteration K = 10. For the rest, we follow the similar setup of Sim CLR (Chen et al., 2020) and Ro CL (Kim et al., 2020). For optimization, we train the encoder with 500 epochs under Adam (Kingma and Ba, 2015) optimizer with the learning rate of 0.001. For the learning rate scheduling, the learning rate is dropped by a factor of 10 for every 100 epochs. The batch size in training is 256. For probabilistic verification (Table 1), we set K = 10, noise level σ = 0.1, number of Gaussian samples NG = 256, and temperature parameter τ = 0.1. |