For self-supervised learning, Rationality implies generalization, provably
Authors: Yamini Bansal, Gal Kaplun, Boaz Barak
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We also conduct an extensive empirical study of the generalization gap and the quantities used in our assumptions for a variety of self-supervision based algorithms, including Sim CLR, AMDIM and Big Bi GAN, on the CIFAR-10 and Image Net datasets. |
| Researcher Affiliation | Academia | Yamini Bansal Harvard University Gal Kaplun Harvard University Boaz Barak Harvard University |
| Pseudocode | No | The paper describes algorithms conceptually and mathematically but does not include any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | We provide our code and data in an anonymous repository on: http://github.com/ICLR2021-rep-gen/. |
| Open Datasets | Yes | We perform an empirical study of the RRM bound for a wide variety of self-supervised training methods on the Image Net (Deng et al., 2009) and CIFAR-10 (Krizhevsky et al., 2009) training datasets. |
| Dataset Splits | No | The paper frequently mentions 'train' and 'test' data but does not explicitly describe a 'validation dataset split' with specific percentages, counts, or methodology for its experiments. |
| Hardware Specification | No | The paper does not specify the hardware used for running the experiments (e.g., GPU models, CPU types, or cloud instance details). |
| Software Dependencies | No | The paper describes optimization algorithms and hyperparameters but does not list specific software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | Table H.3 Summary of training methods with their hyper-parameters for CIFAR-10; Table H.4 Summary of training methods with their hyper-parameters for Image Net. |