Time-FFM: Towards LM-Empowered Federated Foundation Model for Time Series Forecasting
Authors: Qingxiang Liu, Xu Liu, Chenghao Liu, Qingsong Wen, Yuxuan Liang
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our comprehensive experiments indicate that TIME-FFM outperforms state-of-the-arts and promises effective few-shot and zero-shot forecaster. |
| Researcher Affiliation | Collaboration | Qingxiang Liu1,2 Xu Liu3 Chenghao Liu4 Qingsong Wen5 Yuxuan Liang1 1 The Hong Kong University of Science and Technology (Guangzhou) 2 Institute of Computing Technology Chinese Academy of Sciences 3 National University of Singapore 4 Salesforce AI Research 5 Squirrel AI |
| Pseudocode | Yes | The training procedure of TIME-FFM is elaborated in Algorithm 1. |
| Open Source Code | Yes | The code is available at https://github.com/City Mind Lab/Neur IPS24-Time-FFM/tree/main. |
| Open Datasets | Yes | We evaluate on 8 benchmark datasets from various domains: ETTh1, ETTh2, ETTm1, ETTm2, Electricity, Weather, Exchange, and ILI, which have been widely adopted for evaluating time series forecasting performance. Detailed introduction of implementation and datasets can be found in Appendix A. |
| Dataset Splits | Yes | Detailed descriptions of datasets. The dataset size is organized in (training, validation, test). (8545, 2881, 2881) [for ETTh1] and The round with lowest validation value serves as the optimal round, and then the corresponding model is used for test. |
| Hardware Specification | Yes | All models are implemented on PyTorch with all experiments conducted on NVIDIA A100-80G GPUs. |
| Software Dependencies | No | All models are implemented on PyTorch with all experiments conducted on NVIDIA A100-80G GPUs. The Adam optimizer with the initial learning rate of 10 4 is adopted in the training process. |
| Experiment Setup | Yes | The Adam optimizer with the initial learning rate of 10 4 is adopted in the training process. The lookback window Li is set to 36 for the ILI dataset, and 96 for the others. The future prediction window Fi is set to {24, 36, 48, 60} for the ILI dataset, and {96, 192, 336, 720} for other ones. We adopt the pretrained GPT2-backbone of the first 6 layers as the LM encoder. The local epoch E is set to 1 for all domains. The global round number T is set to 100. V , M, P and H are set to 100, 12, 16, and 8 respectively for all domains. Si is set to 4 for the ILI dataset, and 16 for other ones. |