Prometheus: Taming Sample and Communication Complexities in Constrained Decentralized Stochastic Bilevel Learning
Authors: Zhuqing Liu, Xin Zhang, Prashant Khanduri, Songtao Lu, Jia Liu
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we will first conduct experiments to demonstrate the small variance of our new stochastic gradient estimator. Then, we will compare Prometheus convergence with several baselines. 1) New estimator vs. conventional estimator: ... The results are shown in Fig. 3. 2) Convergence Performance: We verify our theoretical results of Prometheus by conducting experiments on a metalearning problem tested on MNIST (Le Cun et al., 1998) and CIFAR-10 (Krizhevsky et al., 2009) datasets. |
| Researcher Affiliation | Collaboration | 1Department of Electrical and Computer Engineering, The Ohio State University, Columbus, OH, USA 2Department of Statistics, Iowa State University, Ames, IA, USA 3Department of Computer Science, Wayne State University, Detroit, MI, USA 4IBM Research, Thomas J. Watson Research Center, Yorktown Heights, NY, USA. |
| Pseudocode | Yes | Algorithm 1 The Prometheus Algorithm at ith agent. |
| Open Source Code | No | The paper does not provide an explicit statement or a link indicating that the source code for the methodology is openly available. |
| Open Datasets | Yes | We verify our theoretical results of Prometheus by conducting experiments on a metalearning problem tested on MNIST (Le Cun et al., 1998) and CIFAR-10 (Krizhevsky et al., 2009) datasets. ... We use the a9a" dataset from LIBSVM repository, which is publicly available at (Chang & Lin, 2011). We divide the a9a dataset into training, validation, and testing sets, which contain 40%, 40%, and 20% samples, respectively. |
| Dataset Splits | Yes | We divide the a9a dataset into training, validation, and testing sets, which contain 40%, 40%, and 20% samples, respectively. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU models, CPU types, or cloud instance specifications) used for running the experiments. |
| Software Dependencies | No | The paper mentions the use of datasets and a two-hidden-layer neural network, but it does not specify any software dependencies with version numbers (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | We set the constant learning rate α = 0.5, β = 0.5 and mini-batch size q = n = 10, pre-defined parameter K = 10. ... We compare Prometheus with these baselines using a two-hidden-layer neural network with 20 hidden units. ... with the network connection probability pc = 0.5, step sizes α = β = 0.01. |