Awesome
FEDformer (ICML 2022 paper)
Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, Rong Jin, "FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting," in Proc. 39th International Conference on Machine Learning (ICML 2022), Baltimore, Maryland, July 17-23, 2022
Frequency Enhanced Decomposed Transformer (FEDformer) is more efficient than standard Transformer with a linear complexity to the sequence length [paper].
Our empirical studies with six benchmark datasets show that compared with state-of-the-art methods, FEDformer can reduce prediction error by 14.8% and 22.6% for multivariate and univariate time series, respectively.
Frequency Enhanced Attention
 |
|---|
| Figure 1. Overall structure of FEDformer |
 |  |
|---|---|
| Figure 2. Frequency Enhanced Block (FEB) | Figure 3. Frequency Enhanced Attention (FEA) |
Main Results
Get Started
- Install Python>=3.8, PyTorch 1.9.0.
- Download data. You can obtain all the six benchmarks from [Autoformer] or [Informer].
- Train the model. We provide the experiment scripts of all benchmarks under the folder
./scripts. You can reproduce the multivariate and univariate experiment results by running the following shell code separately:
bash ./scripts/run_M.sh
bash ./scripts/run_S.sh
Citation
If you find this repo useful, please cite our paper.
@inproceedings{zhou2022fedformer,
title={{FEDformer}: Frequency enhanced decomposed transformer for long-term series forecasting},
author={Zhou, Tian and Ma, Ziqing and Wen, Qingsong and Wang, Xue and Sun, Liang and Jin, Rong},
booktitle={Proc. 39th International Conference on Machine Learning (ICML 2022)},
location = {Baltimore, Maryland},
pages={},
year={2022}
}
Further Reading
Survey on Transformers in Time Series:
Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, and Liang Sun. "Transformers in time series: A survey." arXiv preprint arXiv:2202.07125 (2022). paper
Contact
If you have any question or want to use the code, please contact tian.zt@alibaba-inc.com or maziqing.mzq@alibaba-inc.com .
Acknowledgement
We appreciate the following github repos a lot for their valuable code base or datasets:
https://github.com/thuml/Autoformer
https://github.com/zhouhaoyi/Informer2020