Expressiveness of Rectifier Networks
Authors: Xingyuan Pan, Vivek Srikumar
ICML 2016 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, we experimentally compare threshold networks and their much smaller Re LU counterparts with respect to their ability to learn from synthetically generated data. In this section, we study the following question using synthetic data: Given a rectifier network and a threshold network with same decision boundary, can we learn one using the data generated from another using backpropagation? We use randomly constructed two-layer rectifier networks to generate labeled examples. |
| Researcher Affiliation | Academia | Xingyuan Pan XPAN@CS.UTAH.EDU Vivek Srikumar SVIVEK@CS.UTAH.EDU The University of Utah, Salt Lake City, UT 84112, USA |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any statement or link regarding the availability of open-source code for the described methodology. |
| Open Datasets | No | We use randomly constructed two-layer rectifier networks to generate labeled examples. For each network, we generated 10000 examples and 1500 of which are used as test examples. The paper does not provide access information for this synthetic data. |
| Dataset Splits | Yes | For each network, we generated 10000 examples and 1500 of which are used as test examples. Learning rate is selected with cross-validation from {100, 10 1, 10 2, 10 3, 10 4}. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware used for running the experiments. |
| Software Dependencies | No | The paper does not specify any software dependencies with version numbers. |
| Experiment Setup | Yes | Learning rate is selected with cross-validation from {100, 10 1, 10 2, 10 3, 10 4}. L2-regularization coefficient is 10 4. We use early-stopping optimization with a maximum of 1000 epochs. The minibatch size is 20. For the compressed tanh, we set c = 10000. |