Mutual Contrastive Learning for Visual Representation Learning
Authors: Chuanguang Yang, Zhulin An, Linhang Cai, Yongjun Xu3045-3053
AAAI 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results on image classification and transfer learning to object detection show that MCL can lead to consistent performance gains, demonstrating that MCL can guide the network to generate better feature representations. |
| Researcher Affiliation | Academia | 1Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China 2University of Chinese Academy of Sciences, Beijing, China |
| Pseudocode | No | The paper does not contain a pseudocode block or an algorithm block; it provides architectural diagrams in figures 2 and 3. |
| Open Source Code | Yes | Code is available at https://github.com/winycg/MCL. |
| Open Datasets | Yes | Datasets. We use CIFAR-100 (Krizhevsky, Hinton et al. 2009) and Image Net (Deng et al. 2009) datasets for image classification, following the standard data augmentation and preprocessing pipeline (Huang et al. 2017). |
| Dataset Splits | Yes | We use CIFAR-100 (Krizhevsky, Hinton et al. 2009) and Image Net (Deng et al. 2009) datasets for image classification, following the standard data augmentation and preprocessing pipeline (Huang et al. 2017). and The model is finetuned on trainval07+12 and evaluated on test2007 using m AP. |
| Hardware Specification | No | The paper mentions 'GPU memory' and 'GPU computing' in general terms but does not specify any particular hardware details such as exact GPU models, CPU models, or specific cloud instances used for running the experiments. |
| Software Dependencies | No | The paper discusses various algorithms, network architectures (e.g., ResNets, MoCo), and optimizers (SGD), but it does not specify any software libraries or frameworks with their version numbers (e.g., Python, PyTorch, TensorFlow, CUDA versions). |
| Experiment Setup | Yes | Hyper-parameters settings. Following Sim CLR (Chen et al. 2020b), we use τ = 0.1 on CIFAR and τ = 0.07 on Image Net for similarity calibration of LV CL and LICL. In soft losses of LSoft V CL and LSoft ICL, we utilize τ = 0.1 3 = 0.3 on CIFAR-100 and τ = 0.07 3 = 0.21 on Image Net, 3 larger than that of hard losses to smooth similarity distributions. The contrastive embedding size d is 128. For CIFAR-100, we use K = 126 as the number of negative samples due to a batch size of 128. For Image Net, we retrieve one positive and K = 8192 negative embeddings from the memory bank. Training settings. For CIFAR-100, all networks are trained by SGD with a batch size of 128 and a weight decay of 5 10 4. We use a cosine learning rate that starts from 0.1 and gradually decreases to 0 throughout the 300 epochs. For Image Net, all networks are trained by SGD with a batch size of 256 and a weight decay of 1 10 4. The initial learning rate starts at 0.1 and is decayed by a factor of 10 at 30 and 60 epochs within the total 90 epochs. |