First Steps Toward Understanding the Extrapolation of Nonlinear Models to Unseen Domains
Authors: Kefan Dong, Tengyu Ma
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We prove that the family of nonlinear models of the form f(x) = P fi(xi), where fi is an arbitrary function on the subset of features xi, can extrapolate to unseen distributions, if the covariance of the features is well-conditioned. To the best of our knowledge, this is the first result that goes beyond linear models and the bounded density ratio assumption, even though the assumptions on the distribution shift and function class are stylized. |
| Researcher Affiliation | Academia | Kefan Dong Stanford University kefandong@stanford.edu Tengyu Ma Stanford University tengyuma@stanford.edu |
| Pseudocode | No | No structured pseudocode or algorithm blocks were found in the paper. |
| Open Source Code | No | No explicit statement about the release of source code or a link to a code repository was found. |
| Open Datasets | No | The paper discusses conceptual 'training distributions' and 'test distribution' (e.g., in Figure 1), but does not refer to any specific publicly available datasets nor provide access information for any data used. |
| Dataset Splits | No | The paper is theoretical and does not involve empirical experiments with dataset splits. Therefore, no information on training/validation/test splits is provided. |
| Hardware Specification | No | No specific hardware details (such as GPU/CPU models, memory, or processor types) used for running experiments were mentioned in the paper, as it is a theoretical work. |
| Software Dependencies | No | No specific software dependencies with version numbers were mentioned in the paper. |
| Experiment Setup | No | The paper is theoretical and does not describe any empirical experiments, thus no experimental setup details like hyperparameters or training configurations are provided. |