ResNet with one-neuron hidden layers is a Universal Approximator
Authors: Hongzhou Lin, Stefanie Jegelka
NeurIPS 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We begin by empirically exploring the difference between narrow fully connected networks, with d neurons per hidden layer, and Res Net via a simple example: classifying the unit ball in the plane. Figure 2 shows the results. |
| Researcher Affiliation | Academia | Hongzhou Lin MIT Cambridge, MA 02139 hongzhou@mit.edu Stefanie Jegelka MIT Cambridge, MA 02139 stefje@mit.edu |
| Pseudocode | No | The paper describes mathematical constructions and proof steps but does not include any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not contain any explicit statement about releasing source code or provide a link to a code repository for the described methodology. |
| Open Datasets | No | The training set consists of randomly generated samples (zi, yi)i=1 n R2 { 1, 1} with yi = 1 if zi 2 1; 1 if 2 zi 2 3. |
| Dataset Splits | No | The paper describes a 'training set' for its motivating example but does not provide specific details on training/validation/test dataset splits. |
| Hardware Specification | No | The paper describes empirical observations in Section 2 but does not provide any specific hardware details (e.g., GPU/CPU models, memory) used for the experiments. |
| Software Dependencies | No | The paper does not specify any software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow versions) that would be needed to replicate the experiments. |
| Experiment Setup | Yes | As loss, we use the logistic loss 1 n P log(1 + e yi ˆ yi), where ˆyi = f N (zi) is the output of the network on the i-th sample. Decision boundaries obtained by training fully connected networks with width d = 2 per hidden layer (top row) and Res Net (bottom row) with one neuron in the hidden layers on the unit ball classification problem. |