BasisFormer: Attention-based Time Series Forecasting with Learnable and Interpretable Basis
Authors: Zelin Ni, Hang Yu, Shizhan Liu, Jianguo Li, Weiyao Lin
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Through extensive experiments on six datasets, we demonstrate that Basis Former outperforms previous state-of-the-art methods by 11.04% and 15.78% respectively for univariate and multivariate forecasting tasks. |
| Researcher Affiliation | Collaboration | Zelin Ni Shanghai Jiao Tong University Shanghai, China nzl5116190@sjtu.edu.cn Hang Yu Ant Group Hangzhou, China hyu.hugo@antgroup.com |
| Pseudocode | No | The paper describes the algorithms and modules in text and diagrams (Figure 1) but does not include any formal pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code is available at: https://github.com/nzl5116190/Basisformer. |
| Open Datasets | Yes | Datasets: The six datasets used in this study comprise the following: 1) ETT [7], which consists of temperature data of electricity transformers; 2) Electricity, which includes power consumption data of several customers; 3) Exchange [24], containing financial exchange rate within a specific time range; 4) Traffic, comprising data related to road traffic; 5) Weather, which involves various weather indicators; and 6) Illness, consisting of recorded influenza-like illness data. ... All the six datasets can be downloaded from https://drive.google.com/drive/folders/ 1ZOYp TUa82_j Ccx Id Tmyr0LXQfva M9v Iy?usp=sharing |
| Dataset Splits | Yes | The length of the historical input sequence is maintained at 96 (or 36 for the illness dataset), whereas the length of the sequence to be predicted is selected from a range of values, i.e., {96, 192, 336, 720} ({24, 36, 48, 60} for the illness dataset). ... We partitioned the sequence into past and future parts, uniformly dividing them in a 6:4 ratio for all datasets. ... We used a self-supervised approach for training, reserving 10% of the original training data for validating self-supervised performance. |
| Hardware Specification | Yes | The training and testing of Basis Former are conducted on an NVIDIA Ge Force RTX 3090 graphics card with 24268MB of VRAM. |
| Software Dependencies | No | The paper mentions using the 'Adabelief optimizer [27]' but does not specify its version number or any other software dependencies with their respective version numbers. |
| Experiment Setup | Yes | Experimental setup: The length of the historical input sequence is maintained at 96 (or 36 for the illness dataset), whereas the length of the sequence to be predicted is selected from a range of values, i.e., {96, 192, 336, 720} ({24, 36, 48, 60} for the illness dataset). ... During the training process, we adopt the Adabelief optimizer [27] for optimization. We train the model for 30 epochs with the patience of 3 epochs. All experiments are averaged over 5 trials. ... We find that the performance of Basis Former is robust to the weights in front of the terms in (9). Therefore, we set the weights to be one in all our experiments. Sensitivity analysis of the weights in the loss function can be found in the Appendix A.4. |