Structure of universal formulas
Authors: Dmitry Yarotsky
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | In this paper we analyze the essential structural elements of these highly expressive models. We introduce a hierarchy of expressiveness classes connecting the global approximability property to the weaker property of infinite VC dimension, and prove a series of classification results for several increasingly complex functional families. In particular, we introduce a general family of polynomially-exponentially-algebraic functions that, as we prove, is subject to polynomial constraints. As a consequence, we show that fixed-size neural networks with not more than one layer of neurons having transcendental activations (e.g., sine or standard sigmoid) cannot in general approximate functions on arbitrary finite sets. On the other hand, we give examples of functional families, including two-hidden-layer neural networks, that approximate functions on arbitrary finite sets, but fail to do that on the whole domain of definition. |
| Researcher Affiliation | Academia | Dmitry Yarotsky Skoltech d.yarotsky@skoltech.ru |
| Pseudocode | No | The paper focuses on mathematical proofs and theoretical analysis, presenting equations and theorems. It does not include any pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any statements or links regarding the availability of open-source code for the described methodology. |
| Open Datasets | No | This is a theoretical paper that does not involve empirical experiments, dataset evaluation, or training. Therefore, there is no mention of datasets used for training or their public availability. |
| Dataset Splits | No | This paper is theoretical and does not describe any empirical experiments involving data. Consequently, there is no mention of training, validation, or test dataset splits. |
| Hardware Specification | No | This is a theoretical research paper that focuses on mathematical proofs and analysis of functional families. It does not describe any computational experiments that would require hardware specifications. |
| Software Dependencies | No | The paper is theoretical and does not involve computational experiments or specific software implementations that would require listing software dependencies with version numbers. |
| Experiment Setup | No | This is a theoretical paper, and as such, it does not detail any experimental setup, hyperparameters, or training configurations. Its focus is on mathematical properties and proofs. |