Waxing-and-Waning: a Generic Similarity-based Framework for Efficient Self-Supervised Learning
Authors: Sheng Li, Chao Wu, Ao Li, Yanzhi Wang, Xulong Tang, Geng Yuan
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | The experimental results show that SIMWNW effectively reduces the amount of computation costs in self-supervised model training without compromising accuracy. Specifically, SIMWNW yields up to 54% and 51% computation savings in training from scratch and transfer learning tasks, respectively. |
| Researcher Affiliation | Academia | Sheng Li1, Chao Wu3, Ao Li4, Yanzhi Wang3, Xulong Tang1, Geng Yuan2 1University of Pittsburgh 2University of Georgia 3Northeastern University 4University of Arizona |
| Pseudocode | No | The paper describes the framework design and processes in narrative text and through diagrams (Figure 3), but it does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain any explicit statements about releasing open-source code, nor does it provide a link to a code repository for the described methodology. |
| Open Datasets | Yes | We evaluate the proposed SIMWNW framework in both training from scratch and transfer learning tasks. In the training from scratch task, we use three representative datasets CIFAR-10, CIFAR100 (Krizhevsky & Hinton, 2009), and Image Net (Deng et al., 2009). In the transfer learning task, the encoder is pre-trained on the Image Net dataset and then trained on three datasets Stanford Cars, FGVC Aircraft (Maji et al., 2013), and Caltech-UCSD Birds (CUB) (Wah et al., 2011). |
| Dataset Splits | No | The paper specifies the use of a linear evaluation protocol and details like batch size and epochs, but it does not provide explicit information on dataset splits (e.g., training, validation, test percentages or counts) for reproduction beyond implicitly following standard practices for the named datasets. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU models, CPU types, or memory) used for running the experiments. |
| Software Dependencies | No | The paper does not list specific software dependencies (e.g., programming languages, libraries, or frameworks) with their version numbers that were used for the experiments. |
| Experiment Setup | Yes | The batch size in all the experiments is set to 256. For all datasets except CIFAR, we employ Res Net50 as the encoder. For the CIFAR dataset, we use Res Net18 as the encoder. We apply SIMWNW in two representative SSL frameworks for evaluation: Sim Siam and Sim CLR. In our experiments, the similarity threshold is set to 20... The augmented images are resized to dimensions of 224x224, and we opt for a block size of 30x30... for 32x32 images. The model is trained for 800 epochs. |