On the Convergence of Stochastic Multi-Objective Gradient Manipulation and Beyond
Authors: Shiji Zhou, Wenpeng Zhang, Jiyan Jiang, Wenliang Zhong, Jinjie GU, Wenwu Zhu
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, we empirically verify our proposal in simulation and deep multi-task learning tasks, where we observe comparable or better performance. |
| Researcher Affiliation | Collaboration | Shiji Zhou Tsinghua University zhoushiji00@gmail.com Wenpeng Zhang Ant Group zhangwenpeng0@gmail.com Jiyan Jiang Tsinghua University scjjy95@outlook.com Wenliang Zhong Ant Group yice.zwl@antgroup.com Jinjie Gu Ant Group jinjie.gujj@antgroup.com Wenwu Zhu Tsinghua University wwzhu@tsinghua.edu.cn |
| Pseudocode | Yes | Algorithm 1 Correlation-Reduced Stochastic Multi-objective Gradient Manipulation (CR-MOGM) |
| Open Source Code | No | The paper does not provide an explicit statement about releasing source code or a direct link to a code repository. |
| Open Datasets | Yes | Multi MNIST dataset [37] with 60K examples. ... City Scapes. |
| Dataset Splits | No | The paper mentions "train and test losses" but does not provide specific percentages, sample counts, or explicit descriptions of how the datasets were split for training, validation, or testing. |
| Hardware Specification | No | The paper does not explicitly describe the specific hardware (e.g., GPU models, CPU types, memory) used for running its experiments. |
| Software Dependencies | No | The paper mentions using SGD and Adam optimizers and specific architectures like Le Net and Seg Net, but does not provide version numbers for any software libraries or dependencies (e.g., Python, PyTorch, TensorFlow versions). |
| Experiment Setup | Yes | We initialize x0 = (0, 0.4) and use SGD as the optimizer. To enable fair comparison, the adaptive stepsize is set with k = 0.006/ k for all the algorithms, and the momentum parameter for smoothing the composite weights is set with the suggested parameter setting in Corollary 1. |