Towards Constituting Mathematical Structures for Learning to Optimize
Authors: Jialin Liu, Xiaohan Chen, Zhangyang Wang, Wotao Yin, Hanqin Cai
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Numerical simulations validate our theoretical findings and demonstrate the superior empirical performance of the proposed L2O model. |
| Researcher Affiliation | Collaboration | 1Alibaba Group (U.S.) Inc, Bellevue, WA, USA 2Department of Electrical and Computer Engineering, University of Texas at Austin, Austin, TX, USA 3Department of Statistics and Data Science and Department of Computer Science, University of Central Florida, Orlando, FL, USA. |
| Pseudocode | No | The paper presents mathematical equations for update rules but does not include formal pseudocode or algorithm blocks. |
| Open Source Code | Yes | The code is available online at https://github.com/xhchrn/MS4L2O. |
| Open Datasets | Yes | All models are trained with 500 minibatches (32,000 optimization problems in total) generated synthetically, but are evaluated on both synthesized testing sets and real-world testing sets. We elaborate more on the data generation in the Appendix. The training set includes 32,000 independent optimization problems and the testing set includes 1,024 independent optimization problems. We extract 1000 patches with size 8x8 at random positions from testing images that are randomly chosen from BSDS500 (Martin et al., 2001). We evaluate L2O optimizers (trained on synthesized datasets) on two real-world datasets from the UCI Machine Learning Repository (Dua & Graff, 2017): (i) Ionosphere containing 351 samples of 34 features, and (ii) Spambase containing 4,061 samples of 57 features. |
| Dataset Splits | No | The paper specifies training and testing sets, but does not provide details about a distinct validation set split or its size. |
| Hardware Specification | Yes | All the experiments are conducted on a workstation equipped with four NVIDIA RTX A6000 GPUs. |
| Software Dependencies | Yes | We used PyTorch 1.12 and CUDA 11.3. |
| Experiment Setup | Yes | More specifically, in all our experiments on the LSTM-based L2O models (including our method and other baseline competitors), we use two-layer LSTM cells with 20 hidden units with sigmoid activation functions. During training, each minibatch contains 64 instances of optimization problems, to which the L2O optimizers will be applied for 100 iterations. The 100 iterations are evenly segmented into 5 periods of 20 iterations. Within each of these, the L2O optimizers are trained with truncated Backpropagation Through Time (BPTT) with an Adam optimizer. All models are trained with 500 minibatches (32,000 optimization problems in total) generated synthetically |