Double Trouble in Double Descent: Bias and Variance(s) in the Lazy Regime
Authors: Stéphane D’Ascoli, Maria Refinetti, Giulio Biroli, Florent Krzakala
ICML 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, we present numerical experiments on a standard deep learning setup to show that our results are relevant to the lazy regime of deep neural networks. |
| Researcher Affiliation | Academia | 1Laboratoire de Physique de l Ecole normale sup erieure, ENS, Universit e PSL, CNRS, Sorbonne Universit e, Universit e de Paris, F-75005 Paris, France. |
| Pseudocode | No | The paper does not contain any pseudocode or algorithm blocks. |
| Open Source Code | Yes | The codes necessary to reproduce the results presented in this paper and obtain new ones are given at: https://github.com/mariaref/Random_Features.git |
| Open Datasets | Yes | We train a 5-layer fully-connected network on the CIFAR-10 dataset. |
| Dataset Splits | No | The paper mentions using the CIFAR-10 dataset and evaluates 'test error' but does not explicitly specify the training/validation/test dataset splits, such as percentages or sample counts for each split. |
| Hardware Specification | No | The paper does not specify any particular hardware (e.g., GPU model, CPU, memory) used for running the experiments. It only states that experiments were performed. |
| Software Dependencies | No | The paper mentions using |
| Experiment Setup | Yes | We train a 5-layer fully-connected network on the CIFAR-10 dataset. We keep only the first ten PCA components of the images, and divide the images in two classes according to the parity of the labels. We perform 105 steps of full-batch gradient descent with the Adam optimizer and a learning rate of 0.1, and scale the weights as prescribed in (Jacot et al., 2018). |