Prototypical Contrastive Learning of Unsupervised Representations
Authors: Junnan Li, Pan Zhou, Caiming Xiong, Steven Hoi
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | PCL outperforms state-of-the-art instance-wise contrastive learning methods on multiple benchmarks with substantial improvement in low-resource transfer learning. Code and pretrained models are available at https://github.com/salesforce/PCL. |
| Researcher Affiliation | Industry | Junnan Li, Pan Zhou, Caiming Xiong, Steven C.H. Hoi Salesforce Research {junnan.li,pzhou,cxiong,shoi}@salesforce.com |
| Pseudocode | Yes | APPENDIX B PSEUDO-CODE FOR PROTOTYPICAL CONTRASTIVE LEARNING Algorithm 1: Prototypical Contrastive Learning. |
| Open Source Code | Yes | Code and pretrained models are available at https://github.com/salesforce/PCL. |
| Open Datasets | Yes | We perform training on the Image Net-1M dataset. A Res Net-50 (He et al., 2016) is adopted as the encoder... We evaluate the learned representation on image classification tasks... on two datasets: Places205 (Zhou et al., 2014) for scene recognition and PASCAL VOC2007 (Everingham et al., 2010) for object classification. |
| Dataset Splits | Yes | Table 2 reports the top-5 accuracy on Image Net validation set. |
| Hardware Specification | Yes | We instead target a more commonly accessible training resource for PCL with 4 NVIDIA-V100 GPUs and approximately 5 days of training. |
| Software Dependencies | No | The paper mentions software such as 'faiss', 'LIBLINEAR', and 'MMdetection' but does not provide specific version numbers for these components. |
| Experiment Setup | Yes | We use SGD as our optimizer, with a weight decay of 0.0001, a momentum of 0.9, and a batch size of 256. We train for 200 epochs, where we warm-up the network in the first 20 epochs by only using the Info NCE loss. The initial learning rate is 0.03, and is multiplied by 0.1 at 120 and 160 epochs. In terms of the hyper-parameters, we set τ = 0.1, α = 10, r = 16000, and number of clusters K = {25000, 50000, 100000}. |