Minimum Width for Universal Approximation
Authors: Sejun Park, Chulhee Yun, Jaeho Lee, Jinwoo Shin
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | In this work, we provide the first definitive result in this direction for networks using the RELU activation functions: The minimum width required for the universal approximation of the Lp functions is exactly max{dx + 1, dy}. We also prove that the same conclusion does not hold for the uniform approximation with RELU, but does hold with an additional threshold activation function. Our proof technique can be also used to derive a tighter upper bound on the minimum width required for the universal approximation using networks with general activation functions. ... We note that all formal proofs of Theorems 1 4 are presented in Appendix. |
| Researcher Affiliation | Academia | Sejun Park Chulhee Yun Jaeho Lee Jinwoo Shin KAIST AI MIT EECS KAIST EE emails: sejun.park@kaist.ac.kr, chulheey@mit.edu, jaeho-lee@kaist.ac.kr, jinwoos@kaist.ac.kr |
| Pseudocode | No | The paper describes abstract constructions like a 'coding scheme' but does not include any formal pseudocode blocks or algorithm listings. |
| Open Source Code | No | The paper does not provide any statement or link regarding the release of open-source code for the described methodology. |
| Open Datasets | No | The paper is theoretical and does not use or describe any datasets for training or evaluation. |
| Dataset Splits | No | The paper is theoretical and does not involve empirical validation, therefore, it does not specify dataset splits for training, validation, or testing. |
| Hardware Specification | No | The paper is theoretical and does not report on experiments, therefore, no hardware specifications are mentioned. |
| Software Dependencies | No | The paper is theoretical and does not describe any implementation details or software dependencies with version numbers. |
| Experiment Setup | No | The paper is theoretical and does not report on experiments, thus no experimental setup details are provided. |