Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
Authors: Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, Armand Joulin
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We validate our findings by achieving 75.3% top-1 accuracy on Image Net with Res Net-50, as well as surpassing supervised pretraining on all the considered transfer tasks. |
| Researcher Affiliation | Collaboration | Mathilde Caron1,2 Ishan Misra2 Julien Mairal1 Priya Goyal2 Piotr Bojanowski2 Armand Joulin2 1 Inria 2 Facebook AI Research |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code: https://github.com/facebookresearch/swav |
| Open Datasets | Yes | We validate our findings by achieving 75.3% top-1 accuracy on Image Net with Res Net-50... We test the generalization of Res Net-50 features trained with Sw AV on Image Net (without labels) by transferring to several downstream vision tasks. In Table 2, we compare the performance of Sw AV features with Image Net supervised pretraining. First, we report the linear classification performance on the Places205 [58], VOC07 [16], and i Naturalist2018 [46] datasets. ... on object detection on VOC07+12 using Faster R-CNN [43] and on COCO [32] |
| Dataset Splits | Yes | We evaluate the features of a Res Net-50 [22] trained with Sw AV on Image Net by two experiments: linear classification on frozen features and semi-supervised learning by finetuning with few labels. ... on the Image Net linear evaluation protocol, we reach 75.3% top-1 accuracy with a standard Res Net-50 ... Table 1: Semi-supervised learning on Image Net with a Res Net-50. We finetune the model with 1% and 10% labels and report top-1 and top-5 accuracies. |
| Hardware Specification | Yes | We train each Res Net-50 on 64 V100 16GB GPUs and a batch size of 4096. |
| Software Dependencies | No | The paper mentions software components like 'LARS', 'cosine learning rate', and 'MLP projection head' but does not specify their version numbers or other software dependencies with versions. |
| Experiment Setup | Yes | Note that we train Sw AV during 800 epochs with large batches (4096). We implement in Sw AV the improvements used in Sim CLR, i.e., LARS [55], cosine learning rate [34, 37] and the MLP projection head [9]. We train each Res Net-50 on 64 V100 16GB GPUs and a batch size of 4096. When Sw AV is trained using 2 160 + 4 96 crops |