First order expansion of convex regularized estimators
Authors: Pierre Bellec, Arun Kuchibhotla
NeurIPS 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We consider first order expansions of convex penalized estimators in highdimensional regression problems with random designs. Our setting includes linear regression and logistic regression as special cases. For a given penalty function h and the corresponding penalized estimator ˆβ, we construct a quantity η, the first order expansion of ˆβ, such that the distance between ˆβ and η is an order of magnitude smaller than the estimation error ˆβ β . In this sense, the first order expansion η can be thought of as a generalization of influence functions from the mathematical statistics literature to regularized estimators in high-dimensions. Such first order expansion implies that the risk of ˆβ is asymptotically the same as the risk of η which leads to a precise characterization of the MSE of ˆβ; this characterization takes a particularly simple form for isotropic design. Such first order expansion also leads to inference results based on ˆβ. We provide sufficient conditions for the existence of such first order expansion for three regularizers: the Lasso in its constrained form, the lasso in its penalized form, and the Group-Lasso. The results apply to general loss functions under some conditions and those conditions are satisfied for the squared loss in linear regression and for the logistic loss in the logistic model. |
| Researcher Affiliation | Academia | Pierre C Bellec, Department of Statistics, Rutgers University, 501 Hill Center, Piscataway, NJ 08854, USA. pierre.bellec@rutgers.edu Arun K Kuchibhotla, Department of Statistics, The Wharton School, University of Pennsylvania, Philadelphia, PA 19104, USA. arunku@upenn.edu |
| Pseudocode | No | The paper does not contain any sections or figures explicitly labeled as 'Pseudocode' or 'Algorithm'. Mathematical formulations are presented in equation form. |
| Open Source Code | No | The paper does not provide any explicit statements or links indicating that open-source code for the described methodology is available. |
| Open Datasets | No | This paper focuses on theoretical derivations and proofs for statistical estimators. It does not involve empirical training on specific datasets, nor does it mention any publicly available datasets for experimental evaluation. |
| Dataset Splits | No | As a theoretical paper, there are no empirical experiments that would require specifying training, validation, or test dataset splits. |
| Hardware Specification | No | This paper is theoretical and does not involve running empirical experiments, thus no hardware specifications are mentioned. |
| Software Dependencies | No | This paper is theoretical and does not involve running empirical experiments, thus no specific software dependencies with version numbers are mentioned. |
| Experiment Setup | No | This paper is theoretical and does not describe empirical experiments, therefore no experimental setup details like hyperparameters or training configurations are provided. |