Derivatives of Stochastic Gradient Descent in parametric optimization
Authors: Franck Iutzeler, Edouard Pauwels, Samuel Vaiter
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our theoretical findings are illustrated by numerical experiments on synthetic tasks. |
| Researcher Affiliation | Academia | Franck Iutzeler Université Paul Sabatier, Institut de Mathématiques de Toulouse, France. Edouard Pauwels Toulouse School of Economics, Université Toulouse Capitole, Toulouse, France. Samuel Vaiter CNRS & Université Côte D Azur, Laboratoire J. A. Dieudonné. Nice, France. |
| Pseudocode | No | The paper describes algorithms like SGD and its derivative recursion but does not present them in a formalized pseudocode block or algorithm figure. |
| Open Source Code | Yes | The code is released along with the paper. |
| Open Datasets | Yes | The dataset used is ijcnn1 from libsvm with 49,990 observations and 22 features. |
| Dataset Splits | No | The paper does not explicitly provide details about training, validation, or test dataset splits. |
| Hardware Specification | Yes | All the experiments were performed in jax (Bradbury et al., 2018) on a Mac Book Pro M3 Max. |
| Software Dependencies | No | The paper mentions that experiments were performed in 'jax (Bradbury et al., 2018)' but does not specify a version number for jax or any other software dependencies like numpy or matplotlib mentioned in the supplementary material. |
| Experiment Setup | Yes | We consider various step size regimes and set η0 = µ 4L2 for all experiments. All experiences are performed with m, d = 100, 10 and µ = 0.05. |