Incremental False Negative Detection for Contrastive Learning
Authors: Tsai-Shien Chen, Wei-Chih Hung, Hung-Yu Tseng, Shao-Yi Chien, Ming-Hsuan Yang
ICLR 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments show that our framework outperforms other self-supervised contrastive learning methods on multiple benchmarks in a limited resource setup. |
| Researcher Affiliation | Collaboration | 1National Taiwan University 2Waymo LLC 3University of California, Merced 4Yonsei University 5Google Research |
| Pseudocode | Yes | Finally, we illustrate the framework in Figure 2(b) and present the pseudo code in Appendix B. [...] Algorithm 1: Contrastive Learning with Incremental False Negative Detection |
| Open Source Code | Yes | The source code is available at https://github.com/tsaishien-chen/IFND. |
| Open Datasets | Yes | We apply the proposed method to Image Net (Deng et al., 2009) and CIFAR (Krizhevsky et al., 2009). [...] Image Net (Deng et al., 2009), VOC2007 (Everingham et al., 2010) and Places205 (Zhou et al., 2014) [...] COCO (Lin et al., 2014) |
| Dataset Splits | Yes | We evaluate our representation on the tasks of object detection and semantic segmentation. Following the experiment setting in Mo Co (He et al., 2020) and PCL, we finetune all layers end-to-end on the COCO (Lin et al., 2014) train2017 and evaluate on val2017 dataset. [...] As for semi-supervised learning, we fine-tune the pre-trained Res Net-50 model and the classifier using the subsets of 1% or 10% Image Net labeled training data (note that we use the same subsets of Image Net training data as in Sim CLR). |
| Hardware Specification | Yes | We train the model on 8 Nvidia V100 GPUs for all experiments. |
| Software Dependencies | No | The paper mentions software components like 'Res Net-50', 'SGD optimizer', 'k-means clustering implemented by Johnson et al. (2019)', and other frameworks, but does not provide specific version numbers for any libraries, programming languages, or deep learning frameworks used (e.g., Python, PyTorch, TensorFlow, CUDA). |
| Experiment Setup | Yes | For Image Net pre-training, we follow the same setting in Mo Co v2 (Chen et al., 2020d) and PCL (Li et al., 2021) for fair comparisons. Specifically, we train the encoder for 200 epochs with a mini-batch size of 256 and use the memory-based method (He et al., 2020) to increase the number of contrastive samples in default. [...] In default, we linearly increase the pseudo label acceptance rate from 0% to 100% through the whole training process and use the elimination loss Lelim as the objective. [...] For CIFAR-10 and CIFAR-100 pre-training, we train the encoder for 1000 epochs with a mini-batch size of 1024. [...] we set the initial learning rate to 0.5 and use a linear warm-up strategy for the first 10 epochs. In the rest of the training, we decay the learning rate with the cosine annealing schedule (Loshchilov & Hutter, 2016). For pseudo label assignment, we re-assign the pseudo labels after every 20 training epochs [...] and also use three cluster numbers, {10, 30, 100} for CIFAR-10 and {100, 300, 1000} for CIFAR-100, to hierarchically define semantic concepts. |