FAMO: Fast Adaptive Multitask Optimization
Authors: Bo Liu, Yihao Feng, Peter Stone, Qiang Liu
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct an extensive set of experiments covering multi-task supervised and reinforcement learning problems. Our results indicate that FAMO achieves comparable or superior performance to state-of-the-art gradient manipulation techniques while offering significant improvements in space and computational efficiency. |
| Researcher Affiliation | Collaboration | Bo Liu, Yihao Feng, , Peter Stone, Qiang Liu The University of Texas at Austin, Salesforce AI Research, Sony AI {bliu, pstone, lqiang}@cs.utexas.edu, yihaof@salesforce.com |
| Pseudocode | Yes | Algorithm 1 Fast Adaptive Multitask Optimization (FAMO) |
| Open Source Code | Yes | Code is available at https://github.com/Cranial-XIX/FAMO. |
| Open Datasets | Yes | We consider four supervised benchmarks commonly used in prior MTL research [24, 27, 32, 33]: NYU-v2 [31] (3 tasks), City Scapes [8] (2 tasks), QM-9 [1] (11 tasks), and Celeb A [28] (40 tasks). ... We use 110K molecules from the QM9 example in Py Torch Geometric [10], 10K molecules for validation, and the rest of 10K molecules for testing. |
| Dataset Splits | Yes | We use 110K molecules from the QM9 example in Py Torch Geometric [10], 10K molecules for validation, and the rest of 10K molecules for testing. |
| Hardware Specification | No | The paper does not explicitly describe the hardware used for running its experiments. It states, 'We followed the exact experimental setup from NASHMTL [32]', implying reliance on a referenced paper for such details, but no direct hardware specifications are given within this document. |
| Software Dependencies | No | The paper mentions 'Py Torch Geometric [10]' and 'Soft Actor-Critic (SAC) [15]' but does not provide specific version numbers for these or other software dependencies. |
| Experiment Setup | No | The paper states: 'The experimental setting and hyperparameters all match exactly with those in CAGRAD.' It refers to an external paper for these details rather than providing them directly within the text. |