Awesome
BasisFormer: Attention-based Time Series Forecasting with Learnable and Interpretable Basis
This is the pytorch implementation of Basisformer in the Neurips paper: BasisFormer: Attention-based Time Series Forecasting with Learnable and Interpretable Basis
The architecture of our model(Basisformer) is shown as below:
Through extensive experiments on six datasets, we demonstrate that BasisFormer outperforms previous state-of-the-art methods by 11.04% and 15.78% respectively for univariate and multivariate forecasting tasks.
Setup
1. Create conda environment(Optional)
conda create -n basisformer -y python=3.8
conda activate basisformer
2. Install dependecies
Install the required packages
pip install -r requirements.txt
3. Download the data
We follow the same setting as previous work. The datasets for all the six benchmarks can be obtained from [Autoformer]. The datasets are placed in the 'all_six_datasets' folder of our project. The tree structure of the files are as follows:
Basisformer\all_six_datasets
│
├─electricity
│
├─ETT-small
│
├─exchange_rate
│
├─illness
│
├─traffic
│
└─weather
4. Experimental setup
The length of the historical input sequence is maintained at $96$(or $36$ for the illness dataset), whereas the length of the sequence to be predicted is selected from a range of values, i.e., ${96, 192, 336, 720}$ (${24, 36, 48, 60}$ for the illness dataset). Note that the input length is fixed to be 96 for all methods for a fair comparison. The evaluation is based on the mean squared error (MSE) and mean absolute error (MAE) metrics
Main Results
1. Multivariate Results
2. Univariate Results
Train and Evaluate
1. Multivariate forecasting
sh script/M.sh
2. Univariate forecasting
sh script/S.sh
Note:
If you want to run multiple GPUs in parallel, you can replace script/M.sh and script/S.sh with script/M_parallel.sh and script/S_parallel.sh, respectively.
3. (Optional) Training logs and Weight files
Training logs and weight files can be obtained from Baidu Netdisk(with an extract code: jqsr) or Google Drive
Citation
If you find this repo useful, please cite our paper.
@inproceedings{ni2023basisformer,
title={{Basisformer}: Attention-based Time Series Forecasting with Learnable and Interpretable Basis},
author={Ni, Zelin and Yu, Hang and Liu, Shizhan and Li, Jianguo and Lin, Weiyao},
booktitle={Advances in Neural Information Processing Systems},
year={2023}
}
Contact
If there are any issues, please ask in the GitHub Issue module.
Acknowledgement
We appreciate the following github repos a lot for their valuable code base or datasets:
https://github.com/MAZiqing/FEDformer
https://github.com/thuml/Autoformer
https://github.com/zhouhaoyi/Informer2020