Asymptotics of Wide Networks from Feynman Diagrams
Authors: Ethan Dyer, Guy Gur-Ari
ICLR 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In all cases that were tested empirically, we found that Conjecture 1 holds. For networks with smooth activations, we found that Conjecture 1 always gives a tight bound. For networks with linear or Re LU activations, we always find that the Conjecture holds as an upper bound, but that sometimes the bound is not tight. One such case is highlighted in Table 1. |
| Researcher Affiliation | Industry | Ethan Dyer & Guy Gur-Ari Google Mountain View, CA {edyer,guyga}@google.com |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any explicit statements about releasing source code or links to a code repository. |
| Open Datasets | Yes | All experiments were performed on two-class MNIST, computing a single randomly-chosen component of Θ or f. |
| Dataset Splits | No | The paper mentions training data and mini-batches, but does not provide specific details on how the dataset was split into training, validation, and test sets, or their proportions/counts. |
| Hardware Specification | No | The paper does not explicitly describe the specific hardware (e.g., CPU, GPU models) used for running the experiments. |
| Software Dependencies | No | The paper does not specify software dependencies with version numbers. |
| Experiment Setup | Yes | Sub-figure (a) uses networks trained for 1024 steps with learning rate 1.0 and 1000 samples per class, averaged over 100 initializations. Each curve in figure (b) represents a single instance of the network map evaluated on a random image over the corse of training. The models were trained with 10 samples per class and learning rate 0.1. |