Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Spurious Valleys in One-hidden-layer Neural Network Optimization Landscapes
Authors: Luca Venturi, Afonso S. Bandeira, Joan Bruna
JMLR 2019 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | In this paper, we address this phenomenon by studying a key topological property of the loss: the presence or absence of spurious valleys, defined as connected components of sub-level sets that do not include a global minimum. Focusing on a class of one-hidden-layer neural networks defined by smooth (but generally non-linear) activation functions, we identify a notion of intrinsic dimension and show that it provides necessary and sufficient conditions for the absence of spurious valleys. More concretely, finite intrinsic dimension guarantees that for sufficiently overparametrised models no spurious valleys exist, independently of the data distribution. Conversely, infinite intrinsic dimension implies that spurious valleys do exist for certain data distributions, independently of model overparamatrisation. Besides these positive and negative results, we show that, although spurious valleys may exist in general, they are confined to low risk levels and avoided with high probability on overparametrised models. |
| Researcher Affiliation | Academia | Luca Venturi EMAIL Courant Institute of Mathematical Sciences, 251 Mercer Street, New York, NY 10012 Afonso S. Bandeira EMAIL Joan Bruna EMAIL Courant Institute of Mathematical Sciences, 251 Mercer Street, New York, NY 10012 Center for Data Science, 60 5th Avenue, New York, NY 10011 |
| Pseudocode | No | The paper focuses on mathematical proofs and theoretical analysis, and does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain any statements about open-source code availability, nor does it provide links to repositories or mention code in supplementary materials. |
| Open Datasets | No | The paper analyzes theoretical properties concerning data distributions (e.g., 'generic data distributions', 'Gaussian input distributions') but does not describe experiments performed on specific, named, publicly available datasets. |
| Dataset Splits | No | Since the paper does not present empirical experiments using specific datasets, there is no information provided regarding dataset splits for training, validation, or testing. |
| Hardware Specification | No | The paper is theoretical and focuses on mathematical analysis of neural network optimization landscapes, thus it does not describe any specific hardware used for experiments. |
| Software Dependencies | No | The paper presents theoretical research and does not include details on software dependencies or their specific version numbers as no implementation details or experimental setups are described. |
| Experiment Setup | No | As a theoretical paper focused on mathematical analysis, it does not provide specific experimental setup details such as hyperparameter values, model initialization, or training configurations. |