Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss
Authors: Jeff Z. HaoChen, Colin Wei, Adrien Gaidon, Tengyu Ma
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Minimizing this objective leads to features with provable accuracy guarantees under linear probe evaluation. By standard generalization bounds, these accuracy guarantees also hold when minimizing the training contrastive loss. Empirically, the features learned by our objective can match or outperform several strong baselines on benchmark vision datasets. |
| Researcher Affiliation | Collaboration | Jeff Z. Hao Chen Stanford University Colin Wei Stanford University Adrien Gaidon Toyota Research Institute Tengyu Ma Stanford University |
| Pseudocode | Yes | The pseudo-code for the algorithm and more implementation details can be found in Section A. |
| Open Source Code | No | The paper states 'We implement and test the proposed spectral contrastive loss on standard vision benchmark datasets' but does not provide any explicit statement about releasing the code or a link to a repository. |
| Open Datasets | Yes | We report the accuracy on CIFAR-10/100 [26] and Tiny-Image Net [27] in Table 1. ... In Table 2 we report results on Image Net [18] dataset |
| Dataset Splits | No | The paper mentions a 'linear evaluation protocol' and standard benchmark datasets but does not explicitly provide details about specific training, validation, and test dataset splits or percentages. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used, such as GPU or CPU models. It only mentions 'batch size 384 during pre-training'. |
| Software Dependencies | No | The paper does not specify version numbers for any software dependencies, such as programming languages, libraries, or frameworks used in the experiments. |
| Experiment Setup | Yes | Epochs 200 400 800 (from Table 1 header), We use batch size 384 during pre-training. (from Table 2 caption). |