Loss Landscapes of Regularized Linear Autoencoders
Authors: Daniel Kunin, Jonathan Bloom, Aleksandrina Goeva, Cotton Seed
ICML 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In Section 5, we illustrate these results empirically, with all code and several talks available on Git Hub3. Figure 5. Illustration of the relationship between the eigenvalues of the weight matrix (τ2) and data matrix (σ2) for various values of λ. Points are the empirical and lines are theoretical. The (literal) alignment of theory and practice is visually perfect. |
| Researcher Affiliation | Academia | 1Institute for Computational and Mathematical Engineering, Stanford University, Stanford, California, USA 2Broad Institute of MIT and Harvard, Cambridge, Massachusetts, USA. |
| Pseudocode | Yes | Algorithm 1 LAE PCA |
| Open Source Code | Yes | In Section 5, we illustrate these results empirically, with all code and several talks available on Git Hub3. Footnote 3: github.com/danielkunin/Regularized-Linear-Autoencoders |
| Open Datasets | Yes | See Appendix D.1 for experiments on real data. We have verified that both information and (not surprisingly) symmetric alignment indeed align weights and are competitive with backprop when trained on MNIST and CIFAR10. |
| Dataset Splits | No | The paper mentions using synthetic data and real data (MNIST, CIFAR10) but does not provide specific details on how these datasets were split into training, validation, and test sets. |
| Hardware Specification | No | The paper does not provide specific hardware details (exact GPU/CPU models, processor types with speeds, memory amounts, or detailed computer specifications) used for running its experiments. |
| Software Dependencies | No | The paper mentions using the 'Adam optimizer' but does not provide specific ancillary software details, such as library or solver names with version numbers. |
| Experiment Setup | Yes | In the following experiments, we set k = λ = 10 and fix a data set X R20 20... We train the LAE for each loss using the Adam optimizer for 4000 epochs with random normal initialization, full batch, and learning rate 0.05. |