Efficient Methods for Non-stationary Online Learning
Authors: Peng Zhao, Yan-Feng Xie, Lijun Zhang, Zhi-Hua Zhou
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we provide empirical studies to evaluate our proposed methods. We conduct experiments on the synthetic data. |
| Researcher Affiliation | Academia | Peng Zhao, Yan-Feng Xie, Lijun Zhang, Zhi-Hua Zhou National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China {zhaop, xieyf, zhanglj, zhouzh}@lamda.nju.edu.cn |
| Pseudocode | Yes | Algorithm 1 Efficient Algorithm for Minimizing Dynamic Regret. Algorithm 2 Efficient Algorithm for Problem-dependent Adaptive Regret. |
| Open Source Code | Yes | 3. If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [Yes] |
| Open Datasets | No | We conduct experiments on the synthetic data. ... we control the way to generate the date samples {(xt, yt)}T t=1. Specifically, for t [T], the feature xt is randomly sampled in an Euclidean ball with a diameter D same as the feasible domain of model parameters; and the corresponding label is set as yt = x t w t + εt, where εt is the random noise drawn from [0, 0.1] and w t is the underlying ground-truth model from the feasible domain W generated according to a certain strategy specified below. |
| Dataset Splits | No | The paper uses synthetic data generated by the authors and describes an online learning process. It does not mention or specify any train/validation/test splits for the data. |
| Hardware Specification | Yes | We use a machine with a single CPU (Intel(R) Core(TM) i9-10900K CPU @ 3.70GHz) and 32GB main memory to conduct the experiments. |
| Software Dependencies | No | In the experiment, we use scipy.optimize.Nonlinear Constraint to solve it to perform the projection onto the feasible domain. However, no specific version numbers are provided for scipy or any other software dependencies. |
| Experiment Setup | Yes | In the simulations, we set T = 20000, the domain diameter as D = 6, and the dimension of the domain as d = 8. The feasible domain W is set as an ellipsoid W = w Rd | w Ew λmin(E) (D/2)2 , where E is a certain diagonal matrix and λmin(E) denotes its minimum eigenvalue. ... The step size of random walk is set to be proportional to D/T to ensure a smooth model change. |