MAP IT to Visualize Representations
Authors: Robert Jenssen
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | MAP IT is shown to produce visualizations which capture class structure better than the current state of the art. 1 INTRODUCTION Representation learning is key to any machine learning system. For instance for learning to visualize input representations, or for visualizing learned representations obtained e.g. via deep neural networks trained in a supervised or unsupervised/self-supervised manner in order to gain insight1. ... For that reason, the focus is on a range of relatively modest sized data sets, to convey the basic properties of MAP IT as a new approach. In Appendix C, some further considerations on potential upscaling of the method based on sampling of forces and computation of (entropy) weights (Eq. (19)) are provided. Visualizations below are best viewed in color. Plots are enlarged and repeated in Appendix C, for the benefit of the reader, where also more details about data sets are given. MNIST. A random subset of MNIST consisting of 2000 points is used. Figure 1 has already illustrated that MAP IT, despite not being heavily optimized and by random initialization, produced markedly different representation compared to the alternatives, with a clear class structure and with better separation between classes. |
| Researcher Affiliation | Collaboration | Robert Jenssen Ui T The Arctic University of Norway & U Copenhagen & Norwegian Computing Center |
| Pseudocode | No | The paper provides mathematical derivations and an update rule (Eq. 10) but does not include a formal pseudocode block or an algorithm box. |
| Open Source Code | Yes | MAP IT code is available at https://github.com/SFI-Visual-Intelligence/. |
| Open Datasets | Yes | MNIST. A random subset of MNIST consisting of 2000 points is used. ... Coil 20. This data set (Nene et al., 1996) consists of 1440 greyscale images... Visual Concepts. Images corresponding to three different visual concepts are visualized. SIFT (Lowe, 1999) descriptors... downloaded from the Image Net data base (image-net.org) (Deng et al., 2009). ... Newsgroups. ... Frey faces. The 1965 (28 20) Frey faces are visualized and shown in Fig. 7. ... MNIST, Newsgroups, Frey faces are obtained from http://cs.nyu.edu/ roweis/data.html. |
| Dataset Splits | No | The paper mentions using a random subset of MNIST (2000 points) and performing runs with different learning rates and iterations, but it does not specify explicit training, validation, or testing splits for the datasets used in the experiments. For example, it states: "For each learning rate η of 25, 50 and 100 three runs of MAP IT are performed and in each case the curve of cost versus iterations is shown." |
| Hardware Specification | No | No specific hardware details (e.g., GPU models, CPU types, memory specifications) used for running the experiments were provided in the paper. |
| Software Dependencies | No | The paper mentions software implementations used for comparison (e.g., "For t-SNE the highly optimized Barnes-Hut algorithm is used," "For UMAP, the state-of-the-art implementation from the Herzenberg Lab, Stanford, is used," "The Pac Map author s Python implementation is used"), but it does not specify version numbers for general software dependencies like Python, PyTorch, TensorFlow, or specific libraries beyond their names. |
| Experiment Setup | Yes | In all MAP IT experiments, a random initialization is used, a perplexity value of 15 is used, and no gain or trick are used such as to multiply pjk by some constant in the first (say, 100) iterations (widely done in the t-SNE literature). The common delta-bar-delta rule is used, setting momentum to 0.5 for the first 100 iterations and then 0.8. The learning rate is set to 50 always, over 1000 iterations. |