Acceleration through spectral density estimation
Authors: Fabian Pedregosa, Damien Scieur
ICML 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Through empirical benchmarks on quadratic and logistic regression problems, we identify regimes in which the the proposed methods improve over classical (worst-case) accelerated methods. 7. Experiments We compare the proposed methods and classical accelerated methods on settings with varying degrees of mismatch with our assumptions. We first compare them on quadratics generated from a synthetic dataset, where the empirical spectral density is (approximately) a Marchenko Pastur distribution. We then compare these methods on another quadratic problem, generated using two non-synthetic datasets, where the MP assumption breaks down. Finally, we compare some the applicable methods in a logistic regression problem. |
| Researcher Affiliation | Industry | 1Google Research 2Samsung SAIT AI Lab, Montreal. |
| Pseudocode | Yes | Input: Initial guess x0, λ0 > 0 Algorithm: Iterate over t = 1... xt = xt 1+t 1 t + 1(xt 1 xt 2) λ0 t + 1 f(xt 1) Decaying Exponential Acceleration |
| Open Source Code | No | The paper does not contain any statement about releasing source code or provide links to a code repository. |
| Open Datasets | Yes | We use two UCI datasets: digits2 (n = 1797, d = 64), and breast cancer3 (n = 569, d = 32). 2https://archive.ics.uci.edu/ml/datasets/Optical+Recognition+ of+Handwritten+Digits 3https://archive.ics.uci.edu/ml/datasets/Breast+Cancer+ Wisconsin+%28Diagnostic%29 |
| Dataset Splits | No | The paper mentions using datasets but does not provide specific details on training, validation, or test dataset splits, such as percentages, sample counts, or references to predefined splits. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, processor types, memory amounts) used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific ancillary software details (e.g., library or solver names with version numbers) needed to replicate the experiment. |
| Experiment Setup | No | The paper describes algorithms and their inputs but does not provide specific experimental setup details such as hyperparameters (learning rate, batch size, number of epochs) or specific training configurations used in the empirical evaluations. |