Second-Order Optimization with Lazy Hessians
Authors: Nikita Doikov, El Mahdi Chayti, Martin Jaggi
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate an illustrative numerical experiment on the performance of the proposed second-order methods with lazy Hessian updates. We consider the following convex minimization problem with the Soft Maximum objective (log-sum-exp): min x Rd f(x) := ยต ln n P i=1 exp ai,x bi h ai, x bi i . |
| Researcher Affiliation | Academia | 1Machine Learning and Optimization Laboratory, EPFL, Switzerland. |
| Pseudocode | Yes | Algorithm 1 Cubic Newton with Lazy Hessians; Algorithm 2 Regularized Newton with Lazy Hessians; Algorithm 3 Adaptive Cubic Newton with Lazy Hessians |
| Open Source Code | No | The paper does not contain any statements about releasing source code or links to a code repository for the methodology described. |
| Open Datasets | Yes | Logistic regression: a9a, d = 123, n = 32561, L2-regularization; www.csie.ntu.edu.tw/ cjlin/libsvmtools/datasets/. |
| Dataset Splits | No | The paper does not explicitly provide specific training/test/validation dataset splits (e.g., exact percentages, sample counts, or citations to predefined splits) needed to reproduce the data partitioning. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., exact GPU/CPU models, processor types with speeds, or memory amounts) used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific ancillary software details (e.g., library or solver names with version numbers) needed to replicate the experiment. |
| Experiment Setup | Yes | The regularization parameter is fixed as M := 1. We also show the performance of the Gradient Method as a standard baseline. We use a constant regularization parameter M (correspondingly the stepsize in the Gradient Method), that we choose for each method separately to optimize its performance. |