Towards Gradient-based Bilevel Optimization with Non-convex Followers and Beyond
Authors: Risheng Liu, Yaohua Liu, Shangzhi Zeng, Jin Zhang
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we first verify the theoretical convergence results on non-convex numerical problems compared with existing EG methods and IG methods. Then we test the performance of IAPTT-GM and demonstrate its generalizability to real-world BLO problems with non-convex followers, which are caused by non-convex regularization and neural network structures. In addition, we further validate the performance of the accelerated version (i.e., IA-GM (A)) under LLC with numerical examples and data hyper-cleaning tasks. |
| Researcher Affiliation | Collaboration | Risheng Liu1,2 Yaohua Liu1 Shangzhi Zeng3 Jin Zhang 4,5 1International School of Information Science & Engineering, DUT 2Pazhou Lab, Guangzhou 3Department of Mathematics and Statistics, UVic 4Department of Mathematics, SUSTech 5National Center for Applied Mathematics Shenzhen |
| Pseudocode | Yes | Algorithm 1 The Proposed IAPTT-GM |
| Open Source Code | Yes | The code is available at http://github.com/vis-opt-group/IAPTT-GM. |
| Open Datasets | Yes | We report results of IAPTT-GM and various mainstream methods, e.g., Prototypical Network [48], Relation Net [49] and T-RHG [16] on mini Image Net [47] and tiered Image Net [50] datasets with two different backbones [6, 51] in Table 2. and IAPTT-GM achieves better test performance of both accuracy and F1 score on two datasets, including MNIST [46] and Fashion MNIST [52]. |
| Dataset Splits | Yes | the dataset is randomly split to three disjoint subsets: Dtr for training, Dval for validation and Dtest for testing and Information about the dataset and hyperparameters for numerical experiments can be found in the supplemental material. |
| Hardware Specification | No | The paper states: 'We specify the detailed configuration of computing devices, GPUs in the supplementary materials.', indicating that hardware specifications are not provided in the main text. |
| Software Dependencies | No | The main paper does not explicitly state software dependencies with specific version numbers. Details regarding experimental setup are generally referred to the supplementary materials. |
| Experiment Setup | No | The paper states that 'Information about the dataset and hyperparameters for numerical experiments can be found in the supplemental material.', indicating that specific experimental setup details are not provided in the main text. |