ReLU Network with Width $d+\mathcalO(1)$ Can Achieve Optimal Approximation Rate
Authors: Chenghao Liu, Minghua Chen
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | In this paper, we address this gap by proving that Re LU networks with width d + 1 can achieve the optimal approximation rate for continuous functions over the domain [0, 1]d under Lp norm for p [1, ). We further show that for the uniform norm, a width of d + 11 is sufficient. We also extend the results to narrow feed-forward networks with various activations, confirming their capability to approximate at the optimal rate. This work adds to the understanding of universal approximation of narrow networks. |
| Researcher Affiliation | Academia | 1School of Data Science, City University of Hong Kong. Correspondence to: Minghua Chen <minghua.chen@cityu.edu.hk>. |
| Pseudocode | No | The paper focuses on theoretical proofs and constructions but does not include any pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | No | The paper does not provide any information or links regarding the availability of open-source code for the described methodology. |
| Open Datasets | No | As a theoretical paper, it does not involve training on datasets or mention any public datasets, links, or citations for such use. |
| Dataset Splits | No | The paper is theoretical and does not conduct experiments involving validation dataset splits. |
| Hardware Specification | No | As a theoretical paper, no hardware specifications for running experiments are mentioned. |
| Software Dependencies | No | As a theoretical paper, no specific software dependencies with version numbers are mentioned. |
| Experiment Setup | No | As a theoretical paper, no experimental setup details like hyperparameter values or training configurations are provided. |