Faster Algorithms for Learning Convex Functions
Authors: Ali Siahkamari, Durmus Alp Emre Acar, Christopher Liao, Kelly L Geyer, Venkatesh Saligrama, Brian Kulis
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section we provide experiments on real datasets from the UCI machine learning repository as well as on synthetic datasets. We compare running times between our proposed approach and the baseline interior point method for all three problems (convex regression, DC regression, and Bregman divergence learning). We also compare both our DC regression algorithm as well as our Bregman divergence learning algorithm to state-of-the-art regression and classification methods, and show that our approach is close to state-of-the-art in terms of accuracy. |
| Researcher Affiliation | Academia | 1Boston University, Boston, MA. Correspondence to: Ali Siahkamari <siaa@bu.edu>. |
| Pseudocode | Yes | Algorithm 1 L-update, Algorithm 2 Convex regression, Algorithm 3 Learning a Bregman divergence, Algorithm 4 Difference of convex regression. |
| Open Source Code | Yes | Our code, along with a built-in tuner for our hyperparameter λ and T, is available on our Git Hub repository 1. 1https://github.com/Siahkamari/Piecewise-linear-regression |
| Open Datasets | Yes | In this section we provide experiments on real datasets from the UCI machine learning repository and listing specific datasets like "Parkinson Speech Dataset", "Wine Quality", etc. in Tables 3, 4, and 5. |
| Dataset Splits | Yes | All results are reported either on the pre-specifed train/test split or a 5-fold cross validation set based on instruction for each dataset. We choose λ from a grid 10 3:3 by 5 fold cross validation. |
| Hardware Specification | Yes | For the ADMM-based methods, we use a V100 Nvidia GPU processor with 11 gigabyte of GPU memory, and 4 cores of CPU. For all the other methods we use a 16 core CPU. |
| Software Dependencies | Yes | We have implemented our algorithm using Py Torch (Paszke et al., 2019) and We use the xgboost 1.6.0 package in Chen & Guestrin (2016). |
| Experiment Setup | Yes | We choose λ from a grid 10 3:3 by 5 fold cross validation. Then we do at most 2 more rounds of grid search around the optimal λ at the first round. We fix ρ = 0.01 and choose T by early stopping, i.e., whenever validation error improvement is less than 10 3 after n iterations of ADMM. |