Self-labelling via simultaneous clustering and representation learning
Authors: Asano YM., Rupprecht C., Vedaldi A.
ICLR 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we evaluate the quality of the representations learned by our Self Labelling (Se La) technique. We first test variants of our method, including ablating its components, in order to find an optimal configuration. Then, we compare our results to the state of the art in self-supervised representation learning, where we find that our method is the best among clustering-based techniques and overall state-of-the-art or at least highly competitive in many benchmarks. |
| Researcher Affiliation | Academia | Yuki M. Asano Christian Rupprecht Andrea Vedaldi Visual Geometry Group University of Oxford {yuki,chrisr,vedaldi}@robots.ox.ac.uk |
| Pseudocode | No | The paper describes the algorithm steps (Step 1: representation learning, Step 2: self-labelling) in Section 3.1 but does not present them in a formal pseudocode block or algorithm box. |
| Open Source Code | Yes | Code and models are available1. 1https://github.com/yukimasano/self-label |
| Open Datasets | Yes | For training data we consider Image Net LSVRC-12 (Deng et al., 2009) and other smaller scale datasets. We also test our features by transferring them to MIT Places (Zhou et al., 2014). All of these are standard benchmarks for evaluation in self-supervised learning. CIFAR-10/100 (Krizhevsky et al., 2009) and SVHN (Netzer et al., 2011). |
| Dataset Splits | Yes | The top-1 accuracy of the linear classifier is then measured on the Image Net validation subset by optionally extracting 10 crops for each validation image (four at the corners and one at the center along with their horizontal flips) and averaging the prediction scores before the accuracy is computed or just taking the a centred crop. |
| Hardware Specification | No | The paper mentions 'the use of the University of Oxford Advanced Research Computing (ARC)' but does not provide specific details such as GPU models, CPU types, or memory sizes used for the experiments. |
| Software Dependencies | No | The paper does not explicitly state the specific version numbers for any software dependencies, such as programming languages, libraries, or frameworks used in the implementation. |
| Experiment Setup | Yes | We train all our self-supervised models with SGD and intial learning rate 0.05 for 400 epochs with two learning rate drops where we divide the rate by ten at 150 and 300 and 350 epochs. For the Sinkhorn-Knopp optimization we set λ = 25 as in (Cuturi, 2013). For CIFAR-10/100 and SVHN we train Alex Net architectures on the resized images with batchsize 128, learning rate 0.03. |