CO2: Consistent Contrast for Unsupervised Visual Representation Learning
Authors: Chen Wei, Huiyu Wang, Wei Shen, Alan Yuille
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically, CO2 improves Momentum Contrast (Mo Co) by 2.9% top-1 accuracy on Image Net linear protocol, 3.8% and 1.1% top-5 accuracy on 1% and 10% labeled semi-supervised settings. It also transfers to image classification, object detection, and semantic segmentation on PASCAL VOC. This shows that CO2 learns better visual representations for these downstream tasks. |
| Researcher Affiliation | Academia | Chen Wei1, Huiyu Wang1, Wei Shen2 , Alan Yuille1 1Johns Hopkins University 2Shanghai Jiao Tong University |
| Pseudocode | No | The paper does not contain any pseudocode or algorithm blocks. |
| Open Source Code | No | The paper references third-party open-source tools like Detectron2 but does not explicitly state that the source code for their proposed CO2 method is publicly available or provide a link to it. |
| Open Datasets | Yes | The unsupervised training is performed on the train split of Image Net-1K (Deng et al., 2009) without using label information. |
| Dataset Splits | Yes | Table 1 summaries the single-crop top-1 classification accuracy on the validation set of Image Net-1K. |
| Hardware Specification | No | The paper states 'The batch size is 256 on 4 GPUs.' but does not specify the type or model of the GPUs used, or any other specific hardware details. |
| Software Dependencies | No | The paper mentions software like 'Detectron2' but does not provide specific version numbers for any software dependencies or frameworks used. |
| Experiment Setup | Yes | We use momentum SGD with momentum 0.9 and weight decay 1e-4. The batch size is 256 on 4 GPUs. ... The model is trained for 200 epochs with the initial learning rate of 0.03. The learning rate is multiplied by 0.1 after 120 and 160 epochs for Mo Co v1, while cosine decayed (Loshchilov & Hutter, 2016) for Mo Co v2. ... For the hyper-parameters of the proposed consistency term, we set τcons as 0.04 and α as 10 for the Mo Co v1-based CO2, and τcon as 0.05, α as 0.3 for the Mo Co v2-based variant. |