Fair Resource Allocation in Multi-Task Learning
Authors: Hao Ban, Kaiyi Ji
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments demonstrate that our method can achieve state-of-the-art performance among gradient manipulation methods on a suite of multitask benchmarks in supervised learning and reinforcement learning. Furthermore, we incorporate the idea of α-fairness into the loss functions of various MTL methods. Extensive empirical studies demonstrate that their performance can be significantly enhanced. |
| Researcher Affiliation | Academia | 1Department of Computer Science and Engineering, University at Buffalo, New York, United States. Correspondence to: Kaiyi Ji <kaiyiji@buffalo.edu>. |
| Pseudocode | Yes | Algorithm 1 Fair Grad for MTL |
| Open Source Code | Yes | Code is available at https: //github.com/Opt MN-Lab/fairgrad. |
| Open Datasets | Yes | Celeb A (Liu et al., 2015) is a large-scale face attributes dataset...QM9 (Ramakrishnan et al., 2014) is a widely-used benchmark...NYU-v2 (Silberman et al., 2012) contains...Cityscapes (Cordts et al., 2016) contains... |
| Dataset Splits | Yes | Following (Navon et al., 2022; Liu et al., 2023), we use the example provided in Pytorch Geometric (Fey & Lenssen, 2019), and use 110k molecules for training, 10k for validation, and the rest 10k for testing. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running experiments. |
| Software Dependencies | No | The paper mentions software like Pytorch Geometric, Soft Actor-Critic (SAC), and MTRL codebase, but does not provide specific version numbers for these software dependencies, which are necessary for reproducible descriptions. |
| Experiment Setup | Yes | We train our method for 15 epochs, using Adam optimizer with learning rate 3e-4. The batch size is 256. |