Improving Transferability of Representations via Augmentation-Aware Self-Supervision
Authors: Hankook Lee, Kibok Lee, Kimin Lee, Honglak Lee, Jinwoo Shin
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments demonstrate that our simple idea consistently improves the transferability of representations learned by supervised and unsupervised methods in various transfer learning scenarios. and 4 Experiments Setup. We pretrain the standard Res Net-18 [20] and Res Net-50 on STL10 [23] and Image Net1003 [31, 32], respectively. We use two recent unsupervised representation learning methods as baselines for pretraining: a contrastive method, Mo Co v2 [1, 14], and a non-contrastive method, Sim Siam [13]. |
| Researcher Affiliation | Collaboration | Hankook Lee1 Kibok Lee23 Kimin Lee4 Honglak Lee25 Jinwoo Shin1 1Korea Advanced Institute of Science and Technology (KAIST) 2University of Michigan 3Amazon Web Services 4University of California, Berkeley 5LG AI Research |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | The code is available at https://github.com/hankook/Aug_Self. |
| Open Datasets | Yes | We pretrain the standard Res Net-18 [20] and Res Net-50 on STL10 [23] and Image Net1003 [31, 32], respectively. |
| Dataset Splits | No | The detailed information of datasets and experimental settings is described in the supplementary material. The main text does not specify exact train/validation/test splits for the datasets. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., exact GPU/CPU models or types) used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific ancillary software details with version numbers (e.g., library or solver names with version numbers like Python 3.8, CPLEX 12.4). |
| Experiment Setup | Yes | We pretrain the standard Res Net-18 [20] and Res Net-50 on STL10 [23] and Image Net1003 [31, 32], respectively. We use two recent unsupervised representation learning methods as baselines for pretraining: a contrastive method, Mo Co v2 [1, 14], and a non-contrastive method, Sim Siam [13]. For STL10 and Image Net100, we pretrain networks for 200 and 500 epochs with a batch size of 256, respectively. For supervised pretraining, we pretrain Res Net-50 for 100 epochs with a batch size of 128 on Image Net100. For augmentations, we use random cropping, flipping, color jittering, grayscaling, and Gaussian blurring following Chen and He [13]. In this section, our Aug Self predicts random cropping and color jittering parameters, i.e., AAug Self = {crop, color}, unless otherwise stated. We set λ = 1.0 for STL10 and λ = 0.5 for Image Net100. |