Self-Supervised Learning Through Efference Copies
Authors: Franz Scherr, Qinghai Guo, Timoleon Moraitis
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Here, we present the results of our experimental evaluations that aim to assess the quality of the representations that can be learned with our approach. For this purpose, we considered various image datasets, including CIFAR-10/100 (Krizhevsky et al., 2009), STL-10 (Coates et al., 2011) as well as the Image Net ILSVRC-2012 dataset (Russakovsky et al., 2015), and compare S-TEC against several other SSL algorithms, such as Sim CLR (Chen et al., 2020a), Mo Co v2 (Chen et al., 2020c), BYOL (Grill et al., 2020) and Re LIC (Mitrovic et al., 2021). |
| Researcher Affiliation | Industry | Franz Scherr1* Huawei Technologies franz.scherr@huawei.com Qinghai Guo2 Huawei Technologies guoqinghai@huawei.com Timoleon Moraitis1* Huawei Technologies timoleon.moraitis@huawei.com 1Huawei Zurich Research Center, Switzerland, 2Huawei ACS Lab, Shenzhen, China |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [Yes] Will be in the supplemental material |
| Open Datasets | Yes | For this purpose, we considered various image datasets, including CIFAR-10/100 (Krizhevsky et al., 2009), STL-10 (Coates et al., 2011) as well as the Image Net ILSVRC-2012 dataset (Russakovsky et al., 2015) |
| Dataset Splits | Yes | We follow the same procedure for all experiments, where we first performed SSL and subsequently determine the class prediction accuracy of a linear classifier that is trained on the emergent representations. Table 3: Image Net results (Res Net-50). Method (100 epoch) Top-1 (val.) |
| Hardware Specification | No | The paper mentions compute resources were provided by Huawei's Von Neumann Lab in the acknowledgments, but it does not specify any particular hardware details such as GPU models, CPU types, or memory used for the experiments. The ethics review also notes 'Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [No]'. |
| Software Dependencies | No | The paper refers to various frameworks and algorithms like "Res Net v1 framework", "LARS algorithm", "Faster R-CNN", and "fully convolutional networks", but it does not specify concrete software dependencies with version numbers (e.g., Python version, PyTorch/TensorFlow versions, CUDA versions). |
| Experiment Setup | Yes | If not otherwise stated, SSL was performed for 1,000 epochs. ... We adopted the Res Net v1 framework (He et al., 2016), and used specifically Res Net-18 and Res Net-50 architectures as the feature extractor f... These networks were optimized during SSL by gradient-descent using the adaptive rate scaling of the LARS algorithm (You et al., 2017) with learning rate warmup and decay. |