Effects of Data Geometry in Early Deep Learning
Authors: Saket Tiwari, George Konidaris
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically corroborate our theoretical results using a toy supervised learning problem. Our experiments demonstrate that number of linear regions varies across manifolds and the results hold with changing neural network architectures. We further demonstrate how the complexity of linear regions is different on the low dimensional manifold of images as compared to the Euclidean space, using the Met Faces dataset. |
| Researcher Affiliation | Academia | Saket Tiwari Department of Computer Science Brown University Providence, RI 02906 saket tiwari@brown.edu George Konidaris Department of Computer Science Brown University Providence, RI 02906 |
| Pseudocode | No | The paper describes algorithms in textual paragraphs (e.g., in Section 4.1 'We calculate the exact number of linear regions... by finding the points...'), but it does not present them in a structured pseudocode block or algorithm listing. |
| Open Source Code | Yes | Code: The code used for running all the experiments and generating all figures is provided in the supplemental material. |
| Open Datasets | Yes | We use the example of the Met Faces dataset [Karras et al., 2020a] to illustrate how data lies on a low dimensional manifold. Specifically, we generate images using the pre-trained Style GAN [Karras et al., 2019, 2020b] trained on the curated Met Faces dataset [Karras et al., 2020a]. |
| Dataset Splits | No | The paper mentions training and uses datasets, but does not provide specific training/validation/test splits (e.g., percentages or sample counts) for reproducibility. It describes data generation and use, but not partitioning for training/validation in a standard ML sense. |
| Hardware Specification | No | The paper states, 'This research was conducted using computational resources and services at the Center for Computation and Visualization, Brown University,' but does not provide specific hardware details such as GPU or CPU models, or memory specifications. |
| Software Dependencies | Yes | We use the SciPy [Virtanen et al., 2020] implementation of sequential least squares programming (SLSQP) algorithm [Kraft, 1988] to solve the optimization problems. |
| Experiment Setup | Yes | We use the Adam optimizer [Kingma and Ba, 2015] for training, with a learning rate of 1e-3, for 10000 epochs. The network architecture is a feed-forward neural network with three hidden layers of 20 neurons each, and Re LU activation function. |