Riemannian SAM: Sharpness-Aware Minimization on Riemannian Manifolds
Authors: Jihun Yun, Eunho Yang
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Lastly, we illustrate the superiority of Riemannian SAM in terms of generalization over previous Riemannian optimization algorithms through experiments on knowledge graph completion and machine translation tasks. We conduct two sets of experiments; (i) knowledge graph completion, and (ii) machine translation. |
| Researcher Affiliation | Collaboration | Jihun Yun KAIST arcprime@kaist.ac.kr Eunho Yang KAIST, AITRICS eunhoy@kaist.ac.kr |
| Pseudocode | Yes | We summarize the overall optimization procedure in Algorithm 1. |
| Open Source Code | No | The paper states "We implement our Riemannian SAM upon Geoopt framework [53] written in Py Torch library [54]." but does not explicitly state that the code for their proposed Riemannian SAM is open-sourced or provide a link to it. |
| Open Datasets | Yes | In our experiments, we use two popular benchmark datasets; WN18RR [41] and Fb15k-237 [55]. We use the BLEU score as an evaluation metric on the IWSLT 14 test set and the newstest2013 test set of WMT 14 respectively. |
| Dataset Splits | No | The paper mentions using standard datasets and refers to previous work for hyperparameters but does not explicitly state the specific training/validation/test dataset splits (e.g., percentages or counts) within its text. |
| Hardware Specification | No | The paper mentions "GPU Numbers 4" in Table 4 for machine translation experiments but does not provide specific hardware details such as GPU models (e.g., NVIDIA A100), CPU models, or memory specifications. |
| Software Dependencies | No | The paper mentions using "Geoopt framework [53] written in Py Torch library [54]" but does not provide specific version numbers for these or any other software dependencies. |
| Experiment Setup | Yes | For marginal hyperparameter tuning, we tune the ascent learning rate ρt {10 4, 10 3, 10 2, 10 1} for Riemannian SAM and the other hyperparameters are the same as HYBONET [52] for fair comparisons. Table 3: Hyperparameter configurations for knowledge graph completion. Table 4: Hyperparameter configurations for machine translation. |