Stochastic Online Linear Regression: the Forward Algorithm to Replace Ridge
Authors: Reda Ouhamma, Odalric-Ambrym Maillard, Vianney Perchet
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Last, we provide numerical experiments to illustrate our results and endorse our intuitions. |
| Researcher Affiliation | Collaboration | Reda Ouhamma Univ. Lille, CNRS, Inria, Centrale Lille, UMR 9189 CRISt AL, F-59000 reda.ouhamma@univ-lille.fr Odalric. Maillard Univ. Lille, CNRS, Inria, Centrale Lille, UMR 9189 CRISt AL, F-59000 Vianney. Perchet Criteo, ENSAE, ENS PARIS-SACLAY |
| Pseudocode | Yes | Algorithm 1: Online ridge regression Algorithm 2: The forward algorithm Algorithm 3: OFULf algorithm |
| Open Source Code | Yes | Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [Yes] See Appendix G |
| Open Datasets | No | The paper uses internally generated synthetic data for experiments and does not provide access information for a publicly available or open dataset. |
| Dataset Splits | No | The paper describes online simulation experiments with generated data, and thus does not provide traditional train/validation/test dataset splits. It does not mention any explicit validation split. |
| Hardware Specification | No | The paper mentions that the type of resources used can be found in Appendix G, but no specific hardware details (e.g., GPU/CPU models, memory) are provided in the main text of the paper. |
| Software Dependencies | No | The paper does not provide specific ancillary software details with version numbers (e.g., library or solver names with versions) in the main text. |
| Experiment Setup | Yes | In Figures 2a and 2b we observe the effect of regularization on the performance of ridge and forward regressions in a 5-dimensional regression setting, we vary λ {1/T, 1/ log(T), 1, 10}, sample a zero mean Gaussian noise with σ = 0.1 and draw features uniformly from the unit ball. [...] We consider a 100-dimensional linear bandit with 10 arms, the parameter vector is drawn from the unit ball, actions are such that xt 200. Noise ϵt L= N(0, 10 1), λ = 10 5, δ = 10 3. |