Awesome
Linear-Attention-Mechanism
This repository implementatesLinear-Attention-Mechanism based on PyTorch.
The detailed formula can be seen in the Linear Attention Mechanism: An Efficient Attention for Semantic Segmentation or Multi-stage Attention ResU-Net for Semantic Segmentation of Fine-Resolution Remote Sensing Images.
If our code is helpful to you, please cite
Li R, Jianlin Su, Duan C and Zheng S. Linear Attention Mechanism: An Efficient Attention for Semantic Segmentation[J]. arXiv preprint arXiv:2007.14902, 2020.
苏剑林. (2020, Jul 04). 《线性Attention的探索:Attention必须有个Softmax吗? 》[Blog post]. Retrieved from https://spaces.ac.cn/archives/7546
R. Li, S. Zheng, C. Duan, J. Su and C. Zhang. "Multistage Attention ResU-Net for Semantic Segmentation of Fine-Resolution Remote Sensing Images." in IEEE Geoscience and Remote Sensing Letters, doi: 10.1109/LGRS.2021.3063381.