Provably tuning the ElasticNet across instances
Authors: Maria-Florina F. Balcan, Misha Khodak, Dravyansh Sharma, Ameet Talwalkar
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | We obtain a novel structural result for the Elastic Net which characterizes the loss as a function of the tuning parameters as a piecewise-rational function with algebraic boundaries. We use this to bound the structural complexity of the regularized loss functions and show generalization guarantees for tuning the Elastic Net regression coefficients in the statistical setting. We also consider the more challenging online learning setting, where we show vanishing average expected regret relative to the optimal parameter pair. We further extend our results to tuning classification algorithms obtained by thresholding regression fits regularized by Ridge, LASSO, or Elastic Net. Our results are the first general learning-theoretic guarantees for this important class of problems that avoid strong assumptions on the data distribution. |
| Researcher Affiliation | Academia | Maria-Florina Balcan Mikhail Khodak Dravyansh Sharma Ameet Talwalkar Carnegie Mellon University |
| Pseudocode | Yes | Algorithm 1 Data-driven Regularization () |
| Open Source Code | No | The paper does not contain any explicit statements about releasing source code for its methodology or links to a code repository. It mentions 'the popular Python library scikit-learn [37]' which is a third-party tool, but not its own. |
| Open Datasets | No | The paper is theoretical and does not conduct experiments with specific datasets. It discusses 'supervised dataset (X, y)' conceptually but does not provide access information for any dataset used for training its own models or experiments. |
| Dataset Splits | No | The paper is theoretical and does not conduct experiments with specific dataset splits. It mentions 'validation split' in the context of its problem formulation but does not provide specific details on how to reproduce data partitioning for experiments. |
| Hardware Specification | No | The paper is theoretical and does not describe any specific hardware used for experiments. The authors indicated 'N/A' for questions related to computational resources in the 'If you ran experiments...' section. |
| Software Dependencies | No | The paper mentions 'scikit-learn' in the context of common practices, but it does not list any specific software dependencies with version numbers required to replicate its theoretical findings or any implied computational steps. |
| Experiment Setup | No | The paper is theoretical and does not conduct empirical experiments, therefore it does not provide specific experimental setup details such as hyperparameter values or training configurations. |