Fine-grained Analysis of Stability and Generalization for Stochastic Bilevel Optimization
Authors: Xuelin Zhang, Hong Chen, Bin Gu, Tieliang Gong, Feng Zheng
IJCAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental analysis validates our theoretical findings. Compared with the previous algorithmic stability analysis, our results do not require the re-initialization of the inner-level parameters before each iteration and are suited for more general objective functions. |
| Researcher Affiliation | Academia | 1College of Informatics, Huazhong Agricultural University, Wuhan 430070, China 2Engineering Research Center of Intelligent Technology for Agriculture, Ministry of Education, Wuhan 430070, China 3School of Artificial Intelligence, Jilin University, Jilin 130012, China 4School of Computer Science and Technology, Xi an Jiaotong University, Xi an 710049, China 5Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen 518055, China zhangxuelin@webmail.hzau.edu.cn, chenh@mail.hzau.edu.cn |
| Pseudocode | Yes | Algorithm 1 Computing algorithm of SSGD Algorithm 2 Computing algorithm of TSGD Algorithm 3 Computing algorithm of UD (Bao et al., 2021) |
| Open Source Code | Yes | Implemented codes (including (Bao et al., 2021) for hyperparameter optimization) and data sets (including the MNIST data (Le Cun, 1998) and the Omnilot data (Lake et al., 2015)) are from publicly available sources. |
| Open Datasets | Yes | Implemented codes (including (Bao et al., 2021) for hyperparameter optimization) and data sets (including the MNIST data (Le Cun, 1998) and the Omnilot data (Lake et al., 2015)) are from publicly available sources. |
| Dataset Splits | Yes | Initially, we randomly select 2000, 2000, and 1000 figures for training, validation and testing, respectively. |
| Hardware Specification | Yes | Each experiment is randomly repeated five times on one Ge Force GTX 1660 SUPER GPU, and the average results are reported. |
| Software Dependencies | No | All experiments are implemented in Python on an Intel Core i7 with 32 GB memory. The paper mentions Python but does not specify its version or any other software dependencies with version numbers. |
| Experiment Setup | Yes | Initially, we randomly select 2000, 2000, and 1000 figures for training, validation and testing, respectively. Meanwhile, set the initial batch size as 8, the maximum number of inner iterations as T = 5000, and the number of outer iterations as K = 5000. The initial step sizes for inner and outer minimization problems are 0.01 and 5, respectively. |