Hardness of Learning Neural Networks with Natural Weights
Authors: Amit Daniely, Gal Vardi
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We prove negative results in this regard, and show that for depth-2 networks, and many natural" weights distributions such as the normal and the uniform distribution, most networks are hard to learn. Namely, there is no efficient learning algorithm that is provably successful for most weights, and every input distribution. It implies that there is no generic property that holds with high probability in such random networks and allows efficient learning. |
| Researcher Affiliation | Collaboration | Amit Daniely School of Computer Science and Engineering, The Hebrew University, Jerusalem, Israel and Google Research Tel-Aviv amit.daniely@mail.huji.ac.il Gal Vardi Weizmann Institute of Science gal.vardi@weizmann.ac.il |
| Pseudocode | No | The paper is theoretical, focusing on proofs and hardness results for learning neural networks. It does not include any pseudocode or algorithm blocks describing a computational procedure. |
| Open Source Code | No | The paper does not provide any statements about releasing source code for the described methodology, nor does it include links to a code repository. |
| Open Datasets | No | The paper is theoretical and does not involve empirical training of models on datasets. It discusses 'input distribution D' for theoretical analysis but does not refer to specific, publicly available datasets for training. |
| Dataset Splits | No | The paper is theoretical and does not describe empirical experiments. Therefore, it does not mention training, validation, or test dataset splits. |
| Hardware Specification | No | The paper is theoretical and does not involve running computational experiments that would require specific hardware. Therefore, no hardware specifications are mentioned. |
| Software Dependencies | No | The paper is theoretical and does not describe computational implementations or experiments. Consequently, it does not list any software dependencies with version numbers. |
| Experiment Setup | No | The paper is theoretical and focuses on mathematical proofs and hardness results rather than practical implementations or experiments. As such, it does not provide details on experimental setup, hyperparameters, or training configurations. |