Non-negative Contrastive Learning
Authors: Yifei Wang, Qi Zhang, Yaoyu Guo, Yisen Wang
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically, we show that these advantages enable NCL to outperform CL significantly on feature disentanglement, feature selection, as well as downstream classification tasks. |
| Researcher Affiliation | Academia | 1 School of Mathematical Sciences, Peking University 2 National Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University 3 Institute for Artificial Intelligence, Peking University |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code is available at https://github.com/PKU-ML/non_neg. |
| Open Datasets | Yes | We utilize Res Net-18 as the backbone and train the models on CIFAR-10, CIFAR-100 and Image Net-100 (Deng et al., 2009). |
| Dataset Splits | No | The paper mentions using 'training data' and 'test data' but does not specify explicit percentages or absolute counts for train/validation/test splits. |
| Hardware Specification | No | The paper does not explicitly describe the hardware used to run its experiments. |
| Software Dependencies | No | The paper mentions 'Py Torch style' but does not provide specific version numbers for software components or libraries. |
| Experiment Setup | Yes | We pretrain the models with batch size 256 and weight decay 0.0001. When implementing NCL, we follow the default settings of Sim CLR. ... For CIFAR-10 and CIFAR-100, the projector is a two-layer MLP with hidden dimension 2048 and output dimension 256. And for Image Net-100, the projector is a two-layer MLP with hidden dimension 16384 and output dimension 2048. |