Visualizing the PHATE of Neural Networks
Authors: Scott Gigante, Adam S. Charles, Smita Krishnaswamy, Gal Mishne
NeurIPS 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate that our visualization provides intuitive, detailed summaries of the learning dynamics beyond simple global measures (i.e., validation loss and accuracy), without the need to access validation data. Furthermore, M-PHATE better captures both the dynamics and community structure of the hidden units as compared to visualization based on standard dimensionality reduction methods (e.g., ISOMAP, t-SNE). We demonstrate M-PHATE with two vignettes: continual learning and generalization. |
| Researcher Affiliation | Academia | Scott Gigante Comp. Biol. and Bioinf. Program Yale University New Haven, CT 06511 scott.gigante@yale.edu; Adam S. Charles Princeton Neuroscience Institute Princeton University Princeton, NJ, 08544 adamsc@princeton.edu; Smita Krishnaswamy Depts. of Genetics and Computer Science Yale University New Haven, CT 06520 smita.krishnaswamy@yale.edu; Gal Mishne Halıcıo glu Data Science Institute University of California, San Diego La Jolla, CA 92093 gmishne@ucsd.edu |
| Pseudocode | No | The paper describes the M-PHATE algorithm mathematically and textually but does not include a formal pseudocode block or algorithm listing. |
| Open Source Code | Yes | An implementation of M-PHATE, along with scripts to reproduce the figures in this paper, is available at https://github.com/scottgigante/M-PHATE. |
| Open Datasets | Yes | To demonstrate our visualization, we train a feedforward neural network with 3 layers of 64 hidden units to classify digits in MNIST [27]. |
| Dataset Splits | No | The paper frequently mentions "validation accuracy" and "validation loss," but it does not specify the exact percentages or counts for training, validation, or test splits. For example, it says "reached 96% validation accuracy" but not how the data was split to achieve this. |
| Hardware Specification | No | The paper mentions "high-performance hardware" in the introduction but does not provide any specific details about the GPU models, CPU models, memory, or cloud computing resources used for running the experiments. |
| Software Dependencies | No | The paper mentions optimizers like Adagrad and Adam, but it does not specify any software dependencies with their version numbers (e.g., Python, PyTorch, TensorFlow versions, or other libraries). |
| Experiment Setup | Yes | To demonstrate our visualization, we train a feedforward neural network with 3 layers of 64 hidden units to classify digits in MNIST [...]. The visualization is built on the time trace T evaluated on the network over a single round of training that lasted 300 epochs [...]. We implemented a 2-layer MLP with 400 units in each hidden layer [...]. Each network was trained for 4 epochs before switching to the next task. [...] we train a 3-layer MLP with 128 hidden units in each layer to classify MNIST. |