Unsupervised visualization of image datasets using contrastive learning
Authors: Niklas Böhm, Philipp Berens, Dmitry Kobak
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We show that the resulting 2D embeddings achieve classification accuracy comparable to the state-of-the-art high-dimensional Sim CLR representations, thus faithfully capturing semantic relationships. Using t-Sim CNE, we obtain informative visualizations of the CIFAR-10 and CIFAR-100 datasets, showing rich cluster structure and highlighting artifacts and outliers. |
| Researcher Affiliation | Academia | Jan Niklas B ohm, Philipp Berens & Dmitry Kobak University of T ubingen, Germany {jan-niklas.boehm,philipp.berens,dmitry.kobak}@uni-tuebingen.de |
| Pseudocode | No | The paper describes the model architecture and loss functions using mathematical equations but does not include any pseudocode or algorithm blocks. |
| Open Source Code | Yes | Our code is available at github.com/berenslab/t-simcne (see iclr2023 branch). |
| Open Datasets | Yes | We used CIFAR-10 and CIFAR-100 datasets (Krizhevsky, 2009) for all our experiments. |
| Dataset Splits | Yes | We used CIFAR-10 and CIFAR-100 datasets (Krizhevsky, 2009) for all our experiments. Each dataset consists of n = 60 000 colored and labeled 32 32 images. ... Only training images were used for training; the test set accuracy after training was 92.8% for CIFAR-10 and 72.4% for CIFAR-100 classes. |
| Hardware Specification | Yes | Each experiment was run on a single Ge Force RTX 2080 Ti GPU (the 5000 epoch experiment was run on a V100 GPU). |
| Software Dependencies | Yes | We used our own Py Torch (Paszke et al., 2019, version 1.12.1) implementation of Sim CLR. ... We heavily used Matplotlib 3.6.0 (Hunter, 2007), Num Py 1.23.1 (Harris et al., 2020), and open TSNE 0.6.2 (Poliˇcar et al., 2019), which, in turn, uses Annoy (Bernhardsson, 2013). |
| Experiment Setup | Yes | We optimized the network for 1000 epochs using SGD with momentum 0.9. The initial learning rate was 0.03 b/256 = 0.12, with linear warm-up over ten epochs (from 0 to 0.12) and cosine annealing (Loshchilov & Hutter, 2017) down to 0 for the remaining epochs. We used batch size b = 1024 and the same set of data augmentations as in Chen et al. (2020). |