Direction-oriented Multi-objective Learning: Simple and Provable Stochastic Algorithms
Authors: Peiyao Xiao, Hao Ban, Kaiyi Ji
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments show that our methods achieve competitive or improved performance compared to existing gradient manipulation approaches in a series of tasks on multi-task supervised learning and reinforcement learning. |
| Researcher Affiliation | Collaboration | 1Department of CSE, University at Buffalo 2 Zuoyi Technology |
| Pseudocode | Yes | Algorithm 1 Stochastic Direction-oriented Multi-objective Gradient descent (SDMGrad) |
| Open Source Code | Yes | Code is available at https://github.com/ml-opt-lab/sdmgrad. |
| Open Datasets | Yes | For the supervised learning setting, we evaluate the performance on the Cityscapes [39] and NYUv2 [40] datasets. ... For the reinforcement learning setting, we evaluate the performance on the MT10 benchmarks, which include 10 robot manipulation tasks under the Meta-World environment [43]. |
| Dataset Splits | No | The paper mentions training data and test data for Multi-Fashion+MNIST ('120000 training images and 20000 test images'), but does not provide explicit train/validation/test splits or ratios for all datasets used, nor for the validation set specifically. |
| Hardware Specification | Yes | The experiments on Cityscapes and NYU-v2 are run on RTX 3090 and Tesla V100 GPU, respectively. ... All experiments on MT10 are run on RTX 2080Ti GPU. |
| Software Dependencies | No | The paper mentions using 'the library released by [10]' and 'MTRL codebase[49]' but does not provide specific version numbers for these or other key software dependencies like Python, PyTorch, or TensorFlow. |
| Experiment Setup | Yes | We search the hyperparameter λ {0.1, 0.2, , 1.0} for our SDMGrad method... We train our method for 200 epochs, using Adam optimizer with learning rate 0.0001 for the first 100 epochs and 0.00005 for the rest. The batch size for Cityscapes and NYU-v2 are 8 and 2 respectively. |