Bilevel Optimization: Convergence Analysis and Enhanced Design
Authors: Kaiyi Ji, Junjie Yang, Yingbin Liang
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We further validate our theoretical results and demonstrate the efficiency of bilevel optimization algorithms by the experiments on meta-learning and hyperparameter optimization. The experiments validate our theoretical results for deterministic bilevel optimization, and demonstrate the superior efficiency of stoc Bi O for stochastic bilevel optimization. |
| Researcher Affiliation | Academia | 1Department of Electrical and Computer Engineering, The Ohio State University. |
| Pseudocode | Yes | Algorithm 1 Bilevel algorithms via AID or ITD |
| Open Source Code | Yes | More details can be found in our code via https://github.com/Junjie Yang97/Stoc Bio_hp. |
| Open Datasets | Yes | We conduct experiments over a 5-way 5-shot task on two datasets: FC100 and mini Image Net. |
| Dataset Splits | No | The paper mentions using "training set", "held-out test sets", "validation and training data" but does not provide specific numerical split percentages or sample counts for the datasets (e.g., FC100, mini Image Net, 20 Newsgroup, MNIST) within the main text. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running experiments. |
| Software Dependencies | No | Our implementation applies the function torch.autograd.grad in Py Torch, which automatically determines the size of Jacobians. |
| Experiment Setup | Yes | Due to the space limitations, we provide the model architectures and hyperparameter settings in Appendix A. |