Scaling Laws in Linear Regression: Compute, Parameters, and Data
Authors: Licong Lin, Jingfeng Wu, Sham Kakade, Peter Bartlett, Jason D. Lee
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our theory is consistent with the empirical neural scaling laws and verified by numerical simulation. |
| Researcher Affiliation | Collaboration | Licong Lin UC Berkeley liconglin@berkeley.edu Jingfeng Wu UC Berkeley uuujf@berkeley.edu Sham M. Kakade Harvard University sham@seas.harvard.edu Peter L. Bartlett UC Berkeley and Google Deep Mind peter@berkeley.edu Jason D. Lee Princeton University jasonlee@princeton.edu |
| Pseudocode | No | The paper describes methods in text and mathematical formulas but does not include structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper checklist states: 'Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [No] Justification: Our experiments are numerical simulations and easy to reproduce.' |
| Open Datasets | No | The paper uses self-generated synthetic data based on specified distributions and assumptions, rather than a publicly available dataset. For example: 'We choose the dimension d sufficiently large to approximate the infinite-dimensional case, and the data are generated so that Assumption 1 is satisfied. Moreover, we choose the covariance H Rd d to be diagonal with Hii ia and tr(H) = 1 for some a > 1.' |
| Dataset Splits | No | The paper describes generating data samples (e.g., '(xt, yt)N t=1 are independent samples from P') and discusses averaging results over independent samples of (w*, S) for simulations, but it does not specify explicit training/validation/test data splits or percentages. |
| Hardware Specification | No | The paper does not provide specific details on the hardware used for running the numerical simulations. The NeurIPS checklist states 'Our experiments are numerical simulations and can run with light compute resources' but no specific models or types are mentioned. |
| Software Dependencies | No | The paper does not provide specific software dependencies, libraries, or their version numbers for the numerical simulations. |
| Experiment Setup | Yes | The paper specifies parameters for the numerical simulations: 'Parameters: σ = 1, γ = 0.1.' and 'The expected risk is computed by averaging over 1000 independent samples of (w , S).' |