Adversarial Masking for Self-Supervised Learning
Authors: Yuge Shi, N Siddharth, Philip Torr, Adam R Kosiorek
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate ADIOS with three different SSL objectives: Sim CLR (Chen et al., 2020), BYOL (Grill et al., 2020), and Sim Siam (Chen & He, 2021). Each set of quantitative results is reported as an average over three random trials. We summarise our training setup in Appendix C. |
| Researcher Affiliation | Collaboration | 1University of Oxford 2The University of Edinburgh & The Alan Turing Institute 3Deep Mind. Correspondence to: Yuge Shi <yshi@robots.ox.ac.uk>, Adam Kosiorek <adamrk@deepmind.com>. |
| Pseudocode | No | No structured pseudocode or algorithm blocks are present in the paper. |
| Open Source Code | No | The paper does not provide an explicit statement or link indicating the availability of open-source code for the described methodology. It mentions using "solo-learn (da Costa et al., 2021)" but this is a third-party library, not their own code release. |
| Open Datasets | Yes | We evaluate the performance of ADIOS on STL10, as well as a downsized version of Image Net100 (Tian et al., 2020)... Both Image Net100 and STL10 are derived from Image Net1k (Russakovsky et al., 2015)... We study the downstream performance of models trained on Image Net100-S, on four different datasets including CIFAR10, CIFAR100 (Krizhevsky et al., 2009), Flowers102 (Nilsback & Zisserman, 2008), and i Naturalist (Horn et al., 2018). |
| Dataset Splits | No | The paper does not provide specific percentages or counts for training, validation, and test splits for the datasets used in its experiments. It mentions using certain datasets for evaluation, but the detailed splitting methodology is not provided. |
| Hardware Specification | No | No specific hardware details (e.g., GPU/CPU models, memory specifications) used for running the experiments are mentioned in the paper. |
| Software Dependencies | No | The paper mentions using "solo-learn (da Costa et al., 2021)" but does not specify its version number or the version numbers of any other software dependencies like programming languages or deep learning frameworks (e.g., Python, PyTorch, TensorFlow, CUDA). |
| Experiment Setup | Yes | We summarise our training setup in Appendix C. ... Hyperparameters including optimiser, momentum, scheduler, epochs and batch size are shared across all models, as seen in Tab. 9. ... Refer to Tab. 11 for the values used for these parameters. Table 9: Optimiser SGD Momentum 0.9 Scheduler warmup cosine Epochs 500 Batch size 128. |