Awesome
One Fits All: Power General Time Series Analysis by Pretrained LM (NeurIPS 2023 Spotlight)
Tian Zhou, Peisong Niu, Xue Wang, Liang Sun, Rong Jin, "One Fits All: Power General Time Series Analysis by Pretrained LM,", NeurIPS, 2023. [paper]
The main challenge that blocks the development of pre-trained model for time series analysis is the lack of a large amount of data for training. In this work, we address this challenge by leveraging language or CV models, pre-trained from billions of tokens, for time series analysis. Specifically, we refrain from altering the self-attention and feedforward layers of the residual blocks in the pre-trained language or image model.
<div align="center"><img src=./pic/model_structure.png width=80% /></div>General Time Series Tasks
The proposed method outperforms other models on most tasks, including long-term forecasting, short-term forecasting, classification, anomaly detection, imputation, and few-shot leanring, zero-short learning.
<div align="center"><img src=./pic/main_result.png width=60% /></div>Get Start
- Install Python>=3.8, PyTorch 1.8.1.
- Follow the instructions provided in the respective task folder.
Citation
If you find this repo useful, please cite our paper.
@inproceedings{zhou2023onefitsall,
title={{One Fits All}: Power General Time Series Analysis by Pretrained LM},
author={Tian Zhou, Peisong Niu, Xue Wang, Liang Sun, Rong Jin},
booktitle={NeurIPS},
year={2023}
}
Further Reading
Survey on Transformers in Time Series:
Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, and Liang Sun. "Transformers in time series: A survey.", IJCAI, 2023. [paper]
Contact
If you have any question or want to use the code, please contact tian.zt@alibaba-inc.com or niupeisong.nps@alibaba-inc.com .
Acknowledgement
We appreciate the following github repos a lot for their valuable code base or datasets:
https://github.com/DAMO-DI-ML/ICML2022-FEDformer