Self-Contrastive Learning: Single-Viewed Supervised Contrastive Framework Using Sub-network
Authors: Sangmin Bae, Sungnyun Kim, Jongwoo Ko, Gihun Lee, Seungjong Noh, Se-Young Yun
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate that Self Con learning improves the classification performance of the encoder network, and empirically analyze its advantages in terms of the single-view and the subnetwork. Furthermore, we provide theoretical evidence of the performance increase based on the mutual information bound. For Image Net classification on Res Net-50, Self Con improves accuracy by +0.6% with 59% memory and 48% time of Supervised Contrastive learning, and a simple ensemble of multi-exit outputs boosts performance up to +1.5%. |
| Researcher Affiliation | Collaboration | Sangmin Bae1*, Sungnyun Kim1*, Jongwoo Ko1, Gihun Lee1, Seungjong Noh2, Se-Young Yun1 1Graduate School of Artificial Intelligence, KAIST, 2SK Hynix |
| Pseudocode | No | The paper includes mathematical equations for loss functions but does not present any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code is available at https://github.com/raymin0223/self-contrastive-learning. |
| Open Datasets | Yes | We present the image classification accuracy for standard benchmarks, such as CIFAR-10, CIFAR-100 (Krizhevsky 2009), Tiny-Image Net (Le and Yang 2015), Image Net-100 (Tian, Krishnan, and Isola 2019), and Image Net (Deng et al. 2009) |
| Dataset Splits | No | The paper mentions using standard benchmarks like CIFAR and ImageNet but does not explicitly detail the training, validation, or test split percentages, sample counts, or explicitly state the use of standard predefined splits for reproducibility within the main text. |
| Hardware Specification | No | The paper does not explicitly describe the specific hardware used for its experiments, such as CPU or GPU models, or memory specifications. |
| Software Dependencies | No | The paper does not provide specific version numbers for software dependencies (e.g., Python, PyTorch, TensorFlow, CUDA). |
| Experiment Setup | Yes | We used the same batch size of 1024 and a learning rate of 0.5 as Khosla et al. (2020) did in CIFAR experiments. The complete implementation details and hyperparameter tuning results are presented in Appendix B. |