Smooth Bilevel Programming for Sparse Regularization
Authors: Clarice Poon, Gabriel Peyré
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We perform a numerical benchmark of the convergence speed of our algorithm against state of the art solvers for Lasso, group Lasso, trace norm and linearly constrained problems. These results highlight the versatility of our approach, removing the need to use different solvers depending on the specificity of the ML problem under study. |
| Researcher Affiliation | Academia | Department of mathematical sciences, University of Bath, Bath BA2 7AY, UK cmshp20@bath.ac.uk CNRS and DMA, Ecole Normale Supérieure, PSL University, 45 rue d Ulm, F-75230 PARIS cedex 05, FRANCE, gabriel.peyre@ens.fr |
| Pseudocode | No | The paper describes algorithms but does not include any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | The code to reproduce the results of this article is available online3. 3 https://github.com/gpeyre/2021-Non Cvx Pro |
| Open Datasets | Yes | We tested on 8 datasets from the Libsvm repository4. 4 https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/ |
| Dataset Splits | Yes | Here λ 0 is the regularisation parameter which is typically tuned by cross-validation, and in the limit case λ = 0, (P0) is a constraint problem minβ R(β) under the constraint L(Xβ, y) = 0. |
| Hardware Specification | Yes | All numerics are conducted on 2.4 GHz Quad-Core Intel Core i5 processor with 16GB RAM. |
| Software Dependencies | No | The paper mentions software like L-BFGS, FISTA, SPG/Spa RSA, CELER, etc., but does not provide specific version numbers for any of them. |
| Experiment Setup | No | The paper describes the choice of regularization parameters and general solution methods (L-BFGS, Cholesky solver) but does not provide specific hyperparameter values (e.g., learning rates, batch sizes, number of epochs, or detailed optimizer settings) for the experiments. |