Automatically Learning Compact Quality-aware Surrogates for Optimization Problems
Authors: Kai Wang, Bryan Wilder, Andrew Perrault, Milind Tambe
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct experiments on three different domains where decision-focused learning has been applied: (i) adversarial behavior learning in network security games with a non-convex objective [41], (ii) movie recommendation with a submodular objective [43], and (iii) portfolio optimization problem with a convex quadratic objective [13]. Throughout all the experiments, we compare the performance and the scalability of the surrogate learning (surrogate), two-stage (TS), and decision-focused (DF) learning approaches. Performance is measured in terms of regret, which is defined as the difference between the achieved solution quality and the solution quality if the unobserved parameters θ were observed directly smaller is better. To compare scalability, we show the training time per epoch and inference time. |
| Researcher Affiliation | Academia | Kai Wang Harvard University Cambridge, MA kaiwang@g.harvard.edu Bryan Wilder Harvard University Cambridge, MA bwilder@g.harvard.edu Andrew Perrault Harvard University Cambridge, MA aperrault@g.harvard.edu Milind Tambe Harvard University Cambridge, MA milind_tambe@harvard.edu |
| Pseudocode | No | The paper does not contain any clearly labeled 'Pseudocode' or 'Algorithm' blocks. |
| Open Source Code | Yes | The implementation of this paper can be found in the following link: https://github.com/guaguakai/ surrogate-optimization-learning |
| Open Datasets | Yes | We use Movie Lens [19] as our dataset. The Movie Lens dataset includes 25M ratings over 62,000 movies by 162,000 users. We use historical daily price and volume data from 2004 to 2017 downloaded from the Quandl WIKI dataset [35]. |
| Dataset Splits | Yes | We generate 35 random (ξ, θ) pairs for the training set, 5 for validation, and 10 for testing. 70% of the user groups are used for training, 10% for validation, and 20% for testing. We chronologically split the dataset into training, validation, and test sets with 70%, 10%, and 20% of the data respectively. |
| Hardware Specification | No | The computations in this paper were run on the FASRC Cannon cluster supported by the FAS Division of Science Research Computing Group at Harvard University. |
| Software Dependencies | Yes | All methods are trained using gradient descent with optimizer Adam [25] with learning rate 0.01. the blackbox optimization solver [40] we use in all experiments. [40] is VIRTANEN, P., GOMMERS, R., OLIPHANT, T. E., HABERLAND, M., REDDY, T., COURNAPEAU, D., BUROVSKI, E., PETERSON, P., WECKESSER, W., BRIGHT, J., ET AL. Sci Py 1.0: Fundamental algorithms for scientific computing in Python. Nature Methods 17, 3 (2020), 261 272. |
| Experiment Setup | Yes | All methods are trained using gradient descent with optimizer Adam [25] with learning rate 0.01 and repeated over 30 independent runs to get the average. Each model is trained for at most 100 epochs with early stopping [34] criteria when 3 consecutive non-improving epochs occur on the validation set. The reparameterization size is set to be 10% of the problem size throughout all three examples. |