On Iterative Hard Thresholding Methods for High-dimensional M-Estimation
Authors: Prateek Jain, Ambuj Tewari, Purushottam Kar
NeurIPS 2014 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our empirical results confirm our predictions with respect to the recovery properties of IHT-style algorithms on badly-conditioned sparse recovery problems, as well as demonstrate that these methods can be orders of magnitudes faster than their L1 and greedy counterparts. |
| Researcher Affiliation | Collaboration | Microsoft Research, INDIA University of Michigan, Ann Arbor, USA {prajain,t-purkar}@microsoft.com, tewaria@umich.edu |
| Pseudocode | Yes | Algorithm 1 Iterative Hard-thresholding; Algorithm 2 Two-stage Hard-thresholding |
| Open Source Code | No | The paper mentions comparing with existing algorithms and implementations (e.g., HTP [14], Gra De S [13], L1 projected scaled sub-gradient technique [23], Fo Ba [24]), but it does not provide any concrete access (link, explicit statement of release) to the authors' own source code for the methodology described. |
| Open Datasets | No | The paper describes how data samples were generated for the simulations (e.g., 'Data samples were generated as Zi = (Xi, Yi) where Xi ~ N(0, Ip) and Yi = <θ, Xi> + ξi where ξi ~ N(0, σ^2)'). It does not use or provide concrete access information for a publicly available, pre-existing dataset. |
| Dataset Splits | No | The paper describes how data was generated for simulations but does not specify train/validation/test splits for a fixed dataset, nor does it discuss splitting its generated data into such sets. |
| Hardware Specification | No | The paper mentions that simulations were conducted but does not provide any specific details about the hardware (e.g., GPU/CPU models, memory) used for these experiments. |
| Software Dependencies | No | The paper mentions various algorithms and methods (e.g., HTP, Gra De S, L1, Fo Ba) that were studied, but it does not specify any software names with version numbers (e.g., libraries, frameworks, or programming language versions) used for their implementation or execution. |
| Experiment Setup | Yes | Data: Our problem setting is identical to the one described in the previous section. We fixed a parameter vector θ by choosing s random coordinates and setting them randomly to 1 values. Data samples were generated as Zi = (Xi, Yi) where Xi ~ N(0, Ip) and Yi = <θ, Xi> + ξi where ξi ~ N(0, σ^2). We studied the effect of varying dimensionality p, sparsity s*, sample size n and label noise level σ on the recovery properties of the various algorithms as well as their run times. We chose baseline values of p = 20000, s* = 100, σ = 0.1, n = fo s* log p where fo is the oversampling factor with default value fo = 2. Keeping all other quantities fixed, we varied one of the quantities and generated independent data samples for the experiments. |