Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Statistical Limits of Adaptive Linear Models: Low-Dimensional Estimation and Inference
Authors: Licong Lin, Mufang Ying, Suvrojit Ghosh, Koulik Khamaru, Cun-Hui Zhang
NeurIPS 2023 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we investigate the performance of TALE empirically, and compare it with the ordinary least squares (OLS) estimator, W-decorrelation proposed by Deshpande et al. [13], and the non-asymptotic confidence intervals derived from Theorem 8 in Lattimore et al. [26]. Our simulation set up entails the motivating Example 2.1 of treatment assignment. In our experiments, at stage i, the treatments Ai {0, 1} are assigned on the sign of bθ(i) 1 , where bθ (i) = (bθ(i) 1 , bθ(i) 2 , . . . , bθ(i) d ) is the least square estimate based on all data up to the time point i 1; here, the first coordinate of bθ(i) 1 is associated with treatment assignment. The detailed data generation mechanism can be found in Appendix. From Figure 2 (top) we see that both TALE and W-decorrelation have valid empirical coverage (i.e., they are close to or above the baseline), while the nonasymptotic confidence intervals are overall conservative and the OLS is downwardly biased. In addition, TALE has confidence intervals that are shorter than those of W-decorrelation, which indicates a better estimation performance. Similar observations occur in the high-dimensional model in Figure 2 (bottom), where we find that both the OLS estimator and W-decorrelation are downwardly biased while TALE has valid coverage. |
| Researcher Affiliation | Academia | Licong Lin Department of Statistics University of California, Berkeley EMAIL Mufang Ying Department of Statistics Rutgers University New Brunswick EMAIL Suvrojit Ghosh Department of Statistics Rutgers University New Brunswick EMAIL Koulik Khamaru Department of Statistics Rutgers University New Brunswick EMAIL Cun-Hui Zhang Department of Statistics Rutgers University New Brunswick EMAIL |
| Pseudocode | Yes | Algorithm 1 Centered OLS for k adaptive coordinates (X, y) 1: ˆµad X ad1n n , ˆµnad X nad1n n 2: e Xad = Xad 1nˆµ ad, e Xnad = Xnad 1nˆµ nad 3: Run OLS on centered response vector y y 1n and centered covariate matrix e X = ( e Xad, e Xnad) Rn d; obtain the estimator eθ = (eθ ad, eθ nad) . |
| Open Source Code | Yes | The code is available at https://github.com/licong-lin/low-dim-debias. |
| Open Datasets | No | The paper describes a simulation setup where data is generated for the experiments (e.g., 'We generate a dataset {(xi, yi)}n i=1 that satisfies the assumptions in Theorem 3.4.' in Section B.2) rather than using a publicly available dataset and providing access information for it. |
| Dataset Splits | No | The paper does not explicitly mention validation splits of any dataset or cross-validation for its experiments. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware (e.g., GPU models, CPU types, memory) used to run the experiments. |
| Software Dependencies | No | The paper mentions that the code is available on GitHub, which might contain dependency information, but the paper itself does not explicitly list software dependencies with specific version numbers in its main text or appendices. |
| Experiment Setup | Yes | Section B.1: 'Sample size n = 1000, d = 300. Replication number 20 for each level of adaptivity (k, d).' Section B.2: 'We choose the noise level σ = 0.3 and the probability p = 0.8.' |