Out-of-Distribution Generalization in Kernel Regression
Authors: Abdulkadir Canatar, Blake Bordelon, Cengiz Pehlevan
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We present applications of our theory to real and synthetic datasets and for many kernels. We compare results of our theory applied to Neural Tangent Kernel with simulations of wide networks and show agreement. |
| Researcher Affiliation | Academia | Abdulkadir Canatar Department of Physics Harvard University Cambridge, MA 02138 canatara@g.harvard.edu Blake Bordelon John A. Paulson School of Engineering and Applied Sciences Harvard University Cambridge, MA 02138 blake_bordelon@g.harvard.edu Cengiz Pehlevan John A. Paulson School of Engineering and Applied Sciences Harvard University Cambridge, MA 02138 cpehlevan@g.harvard.edu |
| Pseudocode | Yes | Algorithm 1: Optimizing Training Measure at sample size P |
| Open Source Code | No | The paper mentions using 'Neural Tangents API [29]' and 'JAX [30]', which are third-party tools, but does not provide concrete access to the source code for the methodology described in this paper. |
| Open Datasets | Yes | We perform gradient descent (labeled beneficial) and gradient ascent (labeled detrimental) on M = 1000 MNIST digits 8 s and 9 s |
| Dataset Splits | No | The paper does not provide specific dataset split information (exact percentages, sample counts, citations to predefined splits, or detailed splitting methodology) for training, validation, or test sets. |
| Hardware Specification | No | The paper does not provide specific hardware details (exact GPU/CPU models, processor types with speeds, memory amounts, or detailed computer specifications) used for running its experiments. |
| Software Dependencies | No | The paper mentions 'Neural Tangents API' and 'JAX' but does not specify version numbers for these or any other software dependencies. |
| Experiment Setup | Yes | We use a depth 3 Re LU fully-connected neural network with 2000 hidden units at each layer and its associated Neural Tangent Kernel (NTK) which are computed using the Neural Tangents API [29]. Input dimension, label noise and ridge parameter are D = 120, "2 = 0 and λ = 10 3. |