Implicit Regularization for Group Sparsity
Authors: Jiangyuan Li, Thanh V Nguyen, Chinmay Hegde, Raymond K. W. Wong
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | SIMULATION STUDIES We conduct various experiments on simulated data to support our theory. In Figure 2, we present the recovery error of w on the left, and recovered group magnitudes on the right. |
| Researcher Affiliation | Academia | Texas A&M University New York University {jiangyuanli, raywong}@tamu.edu; thanhng.cs@gmail.com; chinmay.h@nyu.edu |
| Pseudocode | Yes | Algorithm 1 Gradient descent with weight normalization |
| Open Source Code | Yes | Code is available on https://github.com/jiangyuan2li/Implicit-Group-Sparsity |
| Open Datasets | No | The paper explicitly states, 'We conduct various experiments on simulated data to support our theory. Following the model in Section 2, we sample the entries of X i.i.d. using Rademacher random variables and the entries of the noise vector ξ i.i.d. under N(0, σ2).' This indicates data was simulated, not obtained from a publicly available dataset with concrete access information. |
| Dataset Splits | No | The paper uses simulated data but does not specify any training, validation, or test dataset splits. It only mentions the total number of observations (n) and dimension (p) for its simulations (e.g., 'we set n = 150 and p = 300'). |
| Hardware Specification | No | The paper does not provide any specific hardware details such as GPU/CPU models, memory, or computing infrastructure used for running the experiments. |
| Software Dependencies | No | The paper does not specify any software dependencies with version numbers (e.g., Python 3.x, PyTorch 1.x) that were used to run the experiments. |
| Experiment Setup | Yes | In this experiment, we set n = 150 and p = 300. The number of non-zero entries is 9, divided into 3 groups of size 3. We run both Algorithms 1 and 2 with the same initialization α = 10 6. The step size γ on u and decreased step size η on v are both 10 3. |