Eliminating the Invariance on the Loss Landscape of Linear Autoencoders
Authors: Reza Oftadeh, Jiayi Shen, Zhangyang Wang, Dylan Shell
ICML 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | This paper proposes a new loss function for linear autoencoders (LAEs) and analytically identifies the structure of the associated loss surface. ... Additionally, the computational complexity of the loss and its gradients are the same as MSE and, thus, the new loss is not only of theoretical importance but is of practical value, e.g., for low-rank approximation. ... Section 4 presents some experimental results. |
| Researcher Affiliation | Academia | 1Department of Computer Science and Engineering, Texas A&M University, Texas, USA. Correspondence to: Reza Oftadeh <reza.oftadeh@tamu.edu>. |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide concrete access to source code for the methodology described in this paper. |
| Open Datasets | Yes | For the real data, we choose to use MNIST dataset (Le Cun et al., 1998), which includes 60,000 grayscale handwritten digits images, each of 28x28 = 784 pixels. |
| Dataset Splits | No | The paper mentions total dataset sizes (2000 synthetic samples, 60,000 MNIST images) but does not provide specific training/validation/test dataset splits, percentages, or cross-validation details. |
| Hardware Specification | No | The paper does not provide specific hardware details (like GPU/CPU models or memory) used for running its experiments. |
| Software Dependencies | No | The paper mentions using the 'Adam optimizer' but does not provide specific version numbers for any software dependencies or libraries. |
| Experiment Setup | Yes | The weights of networks are initialized to random numbers with a small enough standard deviation (10^-7 in our case). We choose to use the Adam optimizer with a scheduled learning rate (starting from 10^-3 and ending with 10^-6)... p, the number of desired principal components (PCs), is set to 100... ϵ is a manual tolerance threshold (ϵ = 0.01 in our case). |