Deterministic equivalent and error universality of deep random features learning
Authors: Dominik Schröder, Hugo Cui, Daniil Dmitriev, Bruno Loureiro
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We provide extensive numerical evidence for this conjecture, which requires the derivation of closed-form expressions for the layer-wise post-activation population covariances. |
| Researcher Affiliation | Academia | 1Department of Mathematics, ETH Zurich, 8006 Z urich, Switzerland 2Statistical Physics Of Computation lab., Institute of Physics, Ecole Polytechnique F ed erale de Lausanne (EPFL), 1015 Lausanne, Switzerland 3Department of Mathematics, ETH Zurich and ETH AI Center, 8092 Z urich, Switzerland 4D epartement d Informatique, Ecole Normale Sup erieure (ENS) PSL & CNRS, F-75230 Paris cedex 05, France. |
| Pseudocode | No | The paper contains mathematical derivations and formulas, but no clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | A Git Hub repository with the code employed in the present work can be found here. |
| Open Datasets | No | The paper describes generating synthetic data (xµ, yµ) from distributions like N(0d, Ω0) for its experiments but does not provide access information (link, DOI, citation) for a public dataset. For example, "with xµ N(0d, Ω0) independently and yµ = f (xµ) a (potentially random) target function". |
| Dataset Splits | No | The paper conducts numerical simulations and discusses asymptotic regimes (e.g., "d = 500", "α := n/d"), but it does not provide specific training/validation/test dataset splits (e.g., percentages or counts) or reference standard splits. |
| Hardware Specification | No | The paper mentions numerical simulations in specific dimensions (e.g., "d = 500", "d = 1200") but does not provide any details regarding the specific hardware (CPU, GPU models, memory) used for these simulations. |
| Software Dependencies | No | The paper states that a GitHub repository contains the code used (e.g., "A Git Hub repository with the code employed in the present work can be found here."), but it does not explicitly list specific software dependencies with their version numbers in the text. |
| Experiment Setup | No | The paper provides some experimental parameters, such as regularization strength (e.g., "regularization λ = 0.001" or "λ = 0.1"), but it does not offer a comprehensive description of the experimental setup, including all relevant hyperparameters, optimizer settings, or system-level training configurations. |