Multi-label Contrastive Predictive Coding
Authors: Jiaming Song, Stefano Ermon
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our proposed methods on mutual information estimation, knowledge distillation and unsupervised representation learning. |
| Researcher Affiliation | Academia | Jiaming Song Stanford University tsong@cs.stanford.edu Stefano Ermon Stanford University ermon@cs.stanford.edu |
| Pseudocode | Yes | We include the pseudo-code and a PyTorch implementation to α-ML-CPC in Appendix B. |
| Open Source Code | Yes | Our code is available at https://github.com/jiamings/ml-cpc. |
| Open Datasets | Yes | Following the procedure in [47], we evaluate over 13 different student-teacher pairs on CIFAR-100 [30]. |
| Dataset Splits | No | The paper uses standard datasets like CIFAR-10, CIFAR-100, and ImageNet, which have predefined splits. However, it does not explicitly state the percentages for training, validation, or test splits, nor does it explicitly mention the use of a separate validation set in its experimental setup. |
| Hardware Specification | Yes | The time to compute 200 updates on a Nvidia 1080 Ti GPU with the a PyTorch implementation is 1.15 0.06 seconds with CPC and 1.14 0.04 seconds with ML-CPC |
| Software Dependencies | No | The paper mentions 'PyTorch implementation' but does not specify version numbers for PyTorch or any other software dependencies, making it difficult to fully reproduce the software environment. |
| Experiment Setup | Yes | We use the same values for other hyperparameters as those used in the Mo Co-v2 CPC baseline (more details in Appendix C). For MoCo-v2, we used a batch size of 256, a learning rate of 0.03, cosine decay scheduler for 200 epochs, and a temperature of 0.07. |