Connecting Optimization and Regularization Paths
Authors: Arun Suggala, Adarsh Prasad, Pradeep K. Ravikumar
NeurIPS 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we conduct simulations to corroborate our theoretical findings. We use linear regression to empirically verify our results on connecting ridge-regression and gradient descent. We also corroborate our findings on excess risk and optimality of early-stopping rule for gradient descent. |
| Researcher Affiliation | Academia | Arun Sai Suggala Carnegie Mellon University Pittsburgh, PA 15213 asuggala@cs.cmu.edu Adarsh Prasad Carnegie Mellon University Pittsburgh, PA 15213 adarshp@cs.cmu.edu Pradeep Ravikumar Carnegie Mellon University Pittsburgh, PA 15213 pradeepr@cs.cmu.edu |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide concrete access to source code for the methodology described. |
| Open Datasets | No | The paper describes simulating data rather than using a publicly available or open dataset with access information. 'We simulate a linear model by drawing the covariates from an isotropic gaussian X N(0, Ip p) and the response y|x N( T x, σ2)' and 'We construct a classification dataset by drawing covariates X from isotropic gaussian i.e. X N(0, Ip).' |
| Dataset Splits | No | The paper describes how the data was simulated but does not provide specific dataset split information for training, validation, or testing. |
| Hardware Specification | No | The paper does not provide specific hardware details used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific ancillary software details with version numbers. |
| Experiment Setup | Yes | We generate a sequence of iterates by GD with step size 0.01... We fix p = 100 and vary the samples n from 100 to 1500... We run GD with a step size = 0.123 and construct corresponding points on the regularization path ( (t) = t )... We fix the dimension p = 128 and the number of samples to n = 32. |