Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Bilevel Optimization under Unbounded Smoothness: A New Algorithm and Convergence Analysis
Authors: Jie Hao, Xiaochuan Gong, Mingrui Liu
ICLR 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments on hyperrepresentation learning, hyperparameter optimization, and data hyper-cleaning for text classification tasks demonstrate the effectiveness of our proposed algorithm. |
| Researcher Affiliation | Academia | Jie Hao, Xiaochuan Gong, Mingrui Liu Department of Computer Science, George Mason University, Fairfax, VA 22030, USA EMAIL |
| Pseudocode | Yes | Algorithm 1: BO-REP, Algorithm 2: Update Lower, Algorithm 3: Epoch-SGD |
| Open Source Code | Yes | The code is available at https://github.com/Mingrui Liu-ML-Lab/ Bilevel-Optimization-under-Unbounded-Smoothness. |
| Open Datasets | Yes | Hyper-representation experiment is conducted over Amazon Review Dataset (Blitzer et al., 2006), The hyper-cleaning experiments are conducted over the Sentiment140 dataset (Go et al., 2009) for text classification |
| Dataset Splits | Yes | Dval and Dtr denote validation and training sets respectively. |
| Hardware Specification | Yes | To accurately evaluate each algorithm, we use the machine learning framework Py Torch 1.13 to run each algorithm individually on an NVIDIA RTX A6000 graphics card, and record its training and test loss. |
| Software Dependencies | Yes | we use the machine learning framework Py Torch 1.13 |
| Experiment Setup | Yes | We use grid search to tune the lower-level and upper-level step sizes from {0.001, 0.005, 0.01, 0.05, 0.1, 0.5} for all methods. The best combinations of lower-level and upper-level learning rates are (0.05, 0.1) for MAML and ANIL, (0.05, 0.05) for Stoc Bio, (0.1, 0.01) for TTSA, (0.05, 0.05) for SOBA and SABA, (0.05, 0.1) for MA-SOBA, and (0.001, 0.01) for BO-REP. |