Soft Neighbors are Positive Supporters in Contrastive Visual Representation Learning
Authors: Chongjian GE, Jiangliu Wang, Zhan Tong, Shoufa Chen, Yibing Song, Ping Luo
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our soft neighbor contrastive learning method (SNCLR) on standard visual recognition benchmarks, including image classification, object detection, and instance segmentation. The state-of-the-art recognition performance shows that SNCLR is effective in improving feature representations from both Vi T and CNN encoders. |
| Researcher Affiliation | Collaboration | Chongjian Ge1 Jiangliu Wang2 Zhan Tong3 Shoufa Chen1 Yibing Song4 Ping Luo1 1The University of Hong Kong 2The Chinese University of Hong Kong 3Tencent AI Lab 4AI3 Institute, Fudan University |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper states: 'We provide the homepage for this project.' in a footnote, but it does not explicitly state that source code for the described methodology is available, nor does it provide a direct link to a code repository. |
| Open Datasets | Yes | The pretext training is conducted on the Image Net-1k dataset (Russakovsky et al., 2015) without labels. ... We adopt the prevalent COCO datasets (Lin et al., 2014). |
| Dataset Splits | Yes | Following the standard linear classification protocol (Grill et al., 2020), we freeze the parameters of the encoder backbones and additionally train a linear head. |
| Hardware Specification | Yes | We train SNCLR using 32 NVIDIA A100 GPUs with a batch size of 4096. |
| Software Dependencies | No | The paper mentions software tools like 'Torchvision Toolkit (Paszke et al., 2019)', 'timm toolkit (Wightman, 2019)', and 'Dei T (Touvron et al., 2021)', but it does not specify explicit version numbers for these or other key software components. |
| Experiment Setup | Yes | For Res Net encoders, we adopt the LARS optimizer (You et al., 2017) with cosine annealing schedule (Loshchilov & Hutter, 2016) to train the networks for 800 epochs with a warm-up of 10 epochs. The learning rate is scaled linearly with respect to the batch size via lr = 0.3 Batch Size/256 ... The update coefficient of the momentum network branches is set as 0.99. Unless otherwise stated, we utilize 30 soft neighbors to train SNCLR. Different from Res Net, we leverage Adam W optimizer (Loshchilov & Hutter, 2017) to train Vi Ts for 300 epochs with a warm-up of 40 epochs. Following (Chen et al., 2021), the base learning rate is configured as 1.5 10 4. |