ReSSL: Relational Self-Supervised Learning with Weak Augmentation
Authors: Mingkai Zheng, Shan You, Fei Wang, Chen Qian, Changshui Zhang, Xiaogang Wang, Chang Xu
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results show that our proposed Re SSL significantly outperforms the previous stateof-the-art algorithms in terms of both performance and training efficiency. |
| Researcher Affiliation | Collaboration | 1School of Computer Science, Faculty of Engineering, The University of Sydney 2Sense Time Research 3University of Science and Technology of China 4Department of Automation, Tsinghua University, Institute for Artiļ¬cial Intelligence, Tsinghua University (THUAI), Beijing National Research Center for Information Science and Technology (BNRist) 5The Chinese University of Hong Kong |
| Pseudocode | Yes | Algorithm 1: Relational Self-supervised Learning with Weak Augmentation (Re SSL) |
| Open Source Code | Yes | Code is available at https://github.com/KyleZheng1997/Re SSL |
| Open Datasets | Yes | Small Dataset. CIFAR-10 and CIFAR-100 [31]. Medium Dataset. STL-10 [11] and Tiny Image Net [33]. We also performed our algorithm on the large-scale Image Net-1k dataset [14]. PASCAL VOC2007 dataset [20]. |
| Dataset Splits | Yes | There are 50000 training images and 10000 test images. CIFAR-100 is just like the CIFAR-10, except it has 100 classes containing 600 images each. There are 500 training images and 100 testing images per class. STL10 [11] dataset is composed of 96x96 resolution images of 10 classes, 5K labeled training images, 8K validation images, and 100K unlabeled images. The Tiny Image Net dataset is composed of 64x64 resolution images of 200 classes with 100K training images and 10k validation images. |
| Hardware Specification | No | The paper mentions "GPU memory" and "one GPU" but does not provide specific hardware details like exact GPU/CPU models or processor types. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers, such as PyTorch version or specific library versions. |
| Experiment Setup | Yes | The momentum value and memory buffer size are set to 0.99/0.996 and 4096/16384 for small and medium datasets respectively. Moreover, The model is trained using SGD optimizer with a momentum of 0.9 and weight decay of 5e-4. We linear warm up the learning rate for 5 epochs until it reaches 0.06 Batch Size/256, then switch to the cosine decay scheduler [37]. All the models will be trained for 200 epochs. |