Stochastic Gradient Descent under Markovian Sampling Schemes
Authors: Mathieu Even
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Numerical illustration of our theory We present in Appendix G two experiments on synthetic problems, comparing MC-SAG and MC-SGD to gossip-based and token baselines. |
| Researcher Affiliation | Academia | 1Inria ENS Paris. |
| Pseudocode | Yes | Algorithm 1 Markov Chain SAG (MC-SAG) |
| Open Source Code | No | The paper does not provide any explicit statements or links indicating that its source code is publicly available. |
| Open Datasets | No | The paper uses synthetic data generated internally ("For v V, we take fv(x) = ℓ(x, av, bv) for av and bv random variables.") and does not refer to a publicly available dataset with concrete access information. |
| Dataset Splits | No | The paper describes generating synthetic data but does not specify any training, validation, or test dataset splits. |
| Hardware Specification | No | The paper does not explicitly describe the specific hardware used to run its experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers needed to replicate the experiment. |
| Experiment Setup | Yes | We compare our algorithms MC-SGD and MC-SAGA with Walkman (Mao et al., 2020) and decentralized SGD (D-SGD in Figure 1) (Koloskova et al., 2020; Yu et al., 2019) with both randomized gossip communications and fixed gossip matrix. We consider the non-convex loss function ℓ(x, a, b) = (σ(x a) b)2/2 where σ(t) = 1/(1 + exp( t)) as in Mei et al. (2018). For v V, we take fv(x) = ℓ(x, av, bv) for av and bv random variables. |