Self-Supervised Relational Reasoning for Representation Learning
Authors: Massimiliano Patacchiola, Amos J. Storkey
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate the proposed method following a rigorous experimental procedure, using standard datasets, protocols, and backbones. Self-supervised relational reasoning outperforms the best competitor in all conditions by an average 14% in accuracy, and the most recent state-of-the-art model by 3%. |
| Researcher Affiliation | Academia | Massimiliano Patacchiola School of Informatics University of Edinburgh mpatacch@ed.ac.uk Amos Storkey School of Informatics University of Edinburgh a.storkey@ed.ac.uk |
| Pseudocode | Yes | An overview of the model is given in Figure 1 and the pseudo-code in Appendix C (supp. material). |
| Open Source Code | Yes | code released with an open-source license1; 1https://github.com/mpatacchiola/self-supervised-relational-reasoning |
| Open Datasets | Yes | using standard datasets (CIFAR-10, CIFAR-100, CIFAR-100-20, STL-10, tiny-Image Net, Slimage Net) |
| Dataset Splits | Yes | We follow the linear evaluation protocol defined by Kolesnikov et al. (2019) training the backbone for 200 epochs using the unlabeled training set, and then training for 100 epochs a linear classifier on top of the backbone features (without backpropagation in the backbone weights). |
| Hardware Specification | No | The paper does not specify the hardware used for the experiments (e.g., GPU models, CPU types). |
| Software Dependencies | No | We provide a standardized environment implemented in Pytorch (see code in supp. material). (Only Pytorch is mentioned, without a version number, and no other software dependencies with versions are specified.) |
| Experiment Setup | Yes | Hyperparameters (relation learner): mini-batch of 64 images (K = 16 for Res Net32 on tiny-Image Net, K = 25 for Res Net-34 on STL-10, K = 32 for the rest), Adam optimizer with learning rate 10 3, binary cross-entropy loss with focal factor (γ = 2). Relation module: MLP with 256 hidden units (batch-norm + leaky-Re LU) and a single output unit (sigmoid). Aggregation: we used concatenation as it showed to be more effective (see Appenidx B.8, Table 13 supp. material). Augmentations: horizontal flip (50% chance), random crop-resize, conversion to grayscale (20% chance), and color jitter (80% chance). Backbones: Conv-4, Res Net-8/32/56 and Res Net-34 (He et al., 2016). |