Debiased Contrastive Learning
Authors: Ching-Yao Chuang, Joshua Robinson, Yen-Chen Lin, Antonio Torralba, Stefanie Jegelka
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically, the proposed objective consistently outperforms the state-of-the-art for representation learning in vision, language, and reinforcement learning benchmarks. |
| Researcher Affiliation | Academia | Ching-Yao Chuang, Joshua Robinson, Lin Yen-Chen Antonio Torralba, Stefanie Jegelka CSAIL, Massachusetts Institute of Technology Cambridge, MA 02139, USA {cychuang, joshrob, yenchenl, torralba, stefje}@mit.edu |
| Pseudocode | Yes | Figure 3: Pseudocode for debiased objective with M = 1. The implementation only requires a small modification of the code. We can simply extend the code to debiased objective with M > 1 by changing the pos in line 8 with an average of exponentials for M positive samples. |
| Open Source Code | Yes | The code is available at https://github.com/chingyaoc/DCL. |
| Open Datasets | Yes | First, for CIFAR10 [27] and STL10 [7], we implement Sim CLR [2] with Res Net-50 [17] as the encoder architecture and use the Adam optimizer [23] with learning rate 0.001. ... Following [40], we test our approach on Image Net-100, a randomly chosen subset of 100 classes of Imagenet. ... We use the Book Corpus dataset [25]... |
| Dataset Splits | Yes | All the models are trained for 400 epochs and evaluated by training a linear classifier after fixing the learned embedding. ... 10-fold cross validation is used in testing the performance for binary classification tasks (MR, CR, SUBJ, MPQA). |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU/CPU models, processor types, or memory amounts used for running its experiments. |
| Software Dependencies | No | The paper mentions software components such as 'Sim CLR', 'Res Net-50', and 'Adam optimizer', but does not provide specific version numbers for any of them. |
| Experiment Setup | Yes | Following [2], we set the temperature to t = 0.5 and the dimension of the latent vector to 128. All the models are trained for 400 epochs and evaluated by training a linear classifier after fixing the learned embedding. ... use the Adam optimizer [23] with learning rate 0.001. |