Universal Online Convex Optimization with $1$ Projection per Round
Authors: Wenhao Yang, Yibo Wang, Peng Zhao, Lijun Zhang
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To demonstrate the effectiveness of our proposed method, we also conduct empirical experiments, and the results are presented in Appendix E. [...] We repeat the experiments for five times and record the results in Figure 1. We conduct the experiments on a machine with a single CPU (Apple M1 pro) and 16GB memory. We record both regret and running time (in seconds) for all methods. As shown in Figure 1, the running time of our method is comparable to that of Eff Meta Grad, yet it achieves better results for strongly convex functions. |
| Researcher Affiliation | Academia | Wenhao Yang1,2, Yibo Wang1,2, Peng Zhao1,2, Lijun Zhang1,2, 1National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China 2School of Artificial Intelligence, Nanjing University, Nanjing, China |
| Pseudocode | Yes | Algorithm 2 Efficient Algorithm for Universal OCO (and others, like Algorithm 3, 4, 5, 6, 7 in Appendix A) |
| Open Source Code | No | Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [No] Justification: Due to privacy concerns, we do not include the code. |
| Open Datasets | Yes | We conduct experiments on the ijcnn1 dataset from LIBSVM Data [Chang and Lin, 2011, Prokhorov, 2001], where the dimension of features is d = 22. |
| Dataset Splits | No | The paper describes an online learning setting where data samples are received in batches per round. It states 'In each round t [T], the online learner chooses a decision xt X. After submitting the decision, the online learner receives a batch of data samples {(x(i) t , y(i) t )}m i=1 which are sampled from the dataset'. It does not specify fixed training, validation, or test splits for this online learning setup. |
| Hardware Specification | Yes | We conduct the experiments on a machine with a single CPU (Apple M1 pro) and 16GB memory. |
| Software Dependencies | No | The paper mentions 'LIBSVM Data' as the source for the dataset, but does not specify any software dependencies (e.g., programming languages, libraries, frameworks) with version numbers used for implementing or running their algorithms. |
| Experiment Setup | Yes | In our study, we set T = 2000, the domain diameter as D = 20, and the gradient norm upper bound as G = sqrt(22). [...] For exp-concave functions, the online learner suffers a logistic loss: ft(xt) = 1/m * Sum(log(1 + exp(-y(i)t * xt * x(i)t))). Second, for strongly convex functions, we choose the regularized hinge loss: ft(xt) = 1/m * Sum(max(0, 1 - y(i)t * xt * x(i)t)) + λ/2 * ||xt||^2. Third, for general convex functions, the online learner suffers the absolute loss: ft(xt) = 1/m * Sum(||xt * x(i)t - y(i)t||). |