Unsupervised Representation Learning by Invariance Propagation
Authors: Feng Wang, Huaping Liu, Di Guo, Sun Fuchun
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct extensive quantitative experiments to validate the effectiveness of our method, achieving competitive results on object detection and state-of-the-art results on Image Net, Places205, and VOC2007 linear classification, semi-supervised learning, and transfer learning. |
| Researcher Affiliation | Academia | Feng Wang, Huaping Liu , Di Guo, Fuchun Sun Department of Computer Science and Technology, Tsinghua University, China Beijing National Research Center for Information Science and Technology wang-f20@mails.tsinghua.edu.cn,hpliu@tsinghua.edu.cn guodi.gd@gmail.com,fcsun@tsinghua.edu.cn |
| Pseudocode | No | The paper illustrates a process in Figure 1, but it is an 'Illustration of the positive sample discovery algorithm' and not a formal pseudocode block or algorithm listing. |
| Open Source Code | No | The paper does not provide an explicit statement about releasing source code or a direct link to a code repository. |
| Open Datasets | Yes | We train our unsupervised models on the training set of Image Net [7]. We evaluate the quality of the learned representations on extensive downstream tasks, including linear classification on Image Net, Places205 and Pascal VOC, semi-supervised classification on Image Net, transfer learning on seven small scale datasets, and object detection. We also give the ablation study on several critical components of our method. We visualize the embedding statistics and the easy&hard positive neighbourhoods to provide qualitative analysis. For all experiments, we set τ = 0.07 for linear head and τ = 0.2 for MLP head. We set λinv = 0.6, T = 30, k = 4, M = 4096, l = 3, P = 50. We use SGD optimizer with a momentum of 0.9 to optimize our models. The batch size is set to 128 for Image Net. More details can be found in the supplementary material. |
| Dataset Splits | Yes | We report the best top-1, 1-crop accuracy on the held-out evaluation set. ... We report the top-5 accuracy on the held-out validation set. ... We follow the setting of [41] to randomly choose 1% and 10% labeled images from Image Net training set, and fine-tune the pre-trained unsupervised models. |
| Hardware Specification | No | The paper states 'Our method is both effective and reproducible on standard hardware.' and mentions that SimCLR 'requires a large batch size of 4096 to allocate on 128 TPUs,' but does not specify the hardware used for its own experiments beyond 'standard hardware' or give specific GPU/CPU models. |
| Software Dependencies | No | The paper does not specify versions for any software dependencies used in the experiments (e.g., libraries, frameworks, or programming languages). |
| Experiment Setup | Yes | For all experiments, we set τ = 0.07 for linear head and τ = 0.2 for MLP head. We set λinv = 0.6, T = 30, k = 4, M = 4096, l = 3, P = 50. We use SGD optimizer with a momentum of 0.9 to optimize our models. The batch size is set to 128 for Image Net. More details can be found in the supplementary material. |