OptEx: Expediting First-Order Optimization with Approximately Parallelized Iterations
Authors: Yao Shu, Jiongfeng Fang, Ying He, Fei Yu
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, we provide extensive empirical studies, including synthetic functions, reinforcement learning tasks, and neural network training on various datasets, to underscore the substantial efficiency improvements achieved by Opt Ex in practice. |
| Researcher Affiliation | Academia | Yao Shu# , Jiongfeng Fang , Ying Tiffany He , Fei Richard Yu Guangdong Lab of AI and Digital Economy (SZ), China College of Computer Science and Software Engineering, Shenzhen University, China School of Information Technology, Carleton University, Canada |
| Pseudocode | Yes | Algorithm 1: Opt Ex |
| Open Source Code | Yes | Our implementation is available at https://github.com/youyve/Opt Ex. |
| Open Datasets | Yes | Open AI Gym suite [38], CIFAR-10 [41], MNIST [45], Fashion MNIST [46], Shakespeare Corpus, Harry Potter and the Sorcerers Stone |
| Dataset Splits | No | The paper mentions using standard datasets like CIFAR-10 and MNIST, but does not explicitly state the training/validation/test splits used for these datasets. |
| Hardware Specification | Yes | The wallclock time is evaluated on a single NVIDIA RTX 4090 GPU. The wallclock time is evaluated on an AMD EPYC 7763 CPU. |
| Software Dependencies | No | The paper mentions using Deep Q-Network (DQN), Open AI Gym, and jax.vmap, but does not provide specific version numbers for these or other software dependencies. |
| Experiment Setup | Yes | all the baselines...are based on Adam [4] with a learning rate of 0.1, β1 = 0.9, and β2 = 0.999. Hyperparameters, including a learning rate of 0.001, a reward discount factor of 0.95, and a batch size of 256, are applied. based on SGD [1] with a learning rate of 0.001, a batch size of 512. |