Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methods
Authors: Randall Balestriero, Yann LeCun
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Although SSL originated decades ago [7], recent advances have pushed SSL performances beyond expectations [8 10]. Due to those rapid empirical advances, an urgent need for a principled theoretical understanding of those methods has emerged [11, 12]. Studies in this direction often take one of the three following approaches: (i) studying the training dynamics and optimization landscapes of ... (36th Conference on Neural Information Processing Systems (Neur IPS 2022))... We only provide empirical validation that do not require errors bars |
| Researcher Affiliation | Collaboration | Randall Balestriero Meta AI Research, FAIR NYC, USA rbalestriero@meta.com Yann Le Cun Meta AI Research, FAIR, NYU NYC, USA ylecun@meta.com |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code to reproduce the figures is provided in the supplementary materials |
| Open Datasets | No | The paper does not provide concrete access information (specific link, DOI, repository name, formal citation with authors/year, or reference to established benchmark datasets) for a publicly available or open dataset. It describes general input data X and relation matrix G, but not a specific public dataset. |
| Dataset Splits | No | The paper does not provide specific dataset split information for training, validation, or test sets. It mentions parameters like 'N = 256, K = 16, rank(G) = 4' for experiments, but these are not dataset splits. |
| Hardware Specification | No | We only provide empirical validation on toy settings that do not require GPU or cluster computations |
| Software Dependencies | No | The paper does not provide specific ancillary software details with version numbers (e.g., library or solver names with version numbers like Python 3.8, CPLEX 12.4) needed to replicate the experiment. |
| Experiment Setup | Yes | All experiments employed N = 256, K = 16, rank(G) = 4. ... N = 256, K = 32, rank(G) = 8. ... invariance coefficient γ (α fixed at 1) |