Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing
Authors: Jikai Jin, Zhiyuan Li, Kaifeng Lyu, Simon Shaolei Du, Jason D. Lee
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, we conduct numerical experiments to confirm our theoretical findings. ... Experimental results are presented in Section 6 which verify our theoretical findings. |
| Researcher Affiliation | Academia | 1School of Mathematical Sciences, Peking University 2Department of Computer Science, Stanford University 3Department of Computer Science, Princeton University 4Paul G. Allen School of Computer Science and Engineering, University of Washington 5Department of Electrical and Computer Engineering, Princeton University. Correspondence to: Jikai Jin <jinjikai7@gmail.com>. |
| Pseudocode | No | The paper does not contain any pseudocode or algorithm blocks. It describes the GD update rule as a formula: "Ut+1 = Ut \u2212 \u00b5 \u2207f (Ut) = (I + \u00b5Mt)Ut", but this is not formatted as pseudocode. |
| Open Source Code | No | The paper does not provide an explicit statement or link indicating that the source code for the methodology is openly available. |
| Open Datasets | No | The paper states: "The ground truth Z = XX is generated such that the entries of X are i.i.d. standard Gaussian variables. We use the same ground truth throughout our experiments. For i = 1, 2, ..., m, all entries of the measurement Ai Rd d are chosen i.i.d. from the standard Gaussian N(0, 1)." This describes how the data was generated for the experiments but does not provide a public dataset or access information for an existing one. |
| Dataset Splits | No | The paper describes the generation of synthetic data and experimental parameters but does not specify training, validation, or test splits for any dataset, as the data is generated on-the-fly. |
| Hardware Specification | No | The paper does not explicitly describe the hardware used to run its experiments (e.g., specific GPU or CPU models). |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers. |
| Experiment Setup | Yes | Experimental setup. We consider the matrix sensing problem (1) with d = 50, r = 5, \u03b1 \u2208 {1, 0.1, 0.01, 0.001}, m \u2208 {1000, 2000, 5000}. ... The learning rate of GD is set to be \u00b5 = 0.005. |