Home

Awesome

AiATrack

The official PyTorch implementation of our ECCV 2022 paper:

AiATrack: Attention in Attention for Transformer Visual Tracking

Shenyuan Gao, Chunluan Zhou, Chao Ma, Xinggang Wang, Junsong Yuan

[ECVA Open Access] [ArXiv Preprint] [YouTube Video] [Trained Models] [Raw Results] [SOTA Paper List]

Highlight

:bookmark:Brief Introduction

Transformer trackers have achieved impressive advancements recently, where the attention mechanism plays an important role. However, the independent correlation computation in the attention mechanism could result in noisy and ambiguous attention weights, which inhibits further performance improvement. To address this issue, we propose an attention in attention module (named AiA), which enhances appropriate correlations and suppresses erroneous ones by seeking consensus among all correlation vectors. Our AiA module can be readily applied to both self-attention blocks and cross-attention blocks to facilitate feature aggregation and information propagation for visual tracking. Moreover, we propose a streamlined Transformer tracking framework (dubbed AiATrack), by introducing efficient feature reuse and target-background embeddings to make full use of temporal references. Experiments show that our tracker achieves state-of-the-art performance on several tracking benchmarks while running at a real-time speed.

:bookmark:Strong Performance

The proposed AiATrack sets state-of-the-art results on 8 widely used benchmarks. Using ResNet-50 pre-trianed on ImageNet-1k, we can get:

Benchmark (Metrics)AiATrackLeaderboard
LaSOT (AUC / Norm P / P)69.0 / 79.4 / 73.8PWC
LaSOT Extension (AUC / Norm P / P)47.7 / 55.6 / 55.4
TrackingNet (AUC / Norm P / P)82.7 / 87.8 / 80.4PWC
GOT-10k (AO / SR 0.75 / SR 0.5)69.6 / 63.2 / 80.0PWC
NfS30 (AUC)67.9PWC
OTB100 (AUC)69.6PWC
UAV123 (AUC)70.6PWC
VOT2020 (EAO / A / R)0.530 / 0.764 / 0.827

:bookmark:Inference Speed

The proposed AiATrack can run at 38 fps (frames per second) on a single NVIDIA GeForce RTX 2080 Ti.

:bookmark:Training Cost

It takes nearly two days to train our model on 8 NVIDIA GeForce RTX 2080 Ti (each of which has 11GB GPU memory).

:bookmark:Model Complexity

The proposed AiATrack has 15.79M (million) model parameters.

Release

Trained Models (containing the model we trained on four datasets and the model we trained on GOT-10k only) [download zip file]

Raw Results (containing raw tracking results on the datasets we benchmarked in the paper) [download zip file]

Download and unzip these two zip files under AiATrack project path, then both of them can be directly used by our code.

Let's Get Started

Acknowledgement

:heart::heart::heart:Our idea is implemented base on the following projects. We really appreciate their wonderful open-source works!

Citation

If any parts of our paper and code help your research, please consider citing us and giving a star to our repository.

@inproceedings{gao2022aiatrack,
  title={AiATrack: Attention in Attention for Transformer Visual Tracking},
  author={Gao, Shenyuan and Zhou, Chunluan and Ma, Chao and Wang, Xinggang and Yuan, Junsong},
  booktitle={European Conference on Computer Vision},
  pages={146--164},
  year={2022},
  organization={Springer}
}

Contact

If you have any questions or concerns, feel free to open issues or directly contact me through the ways on my GitHub homepage. Suggestions and collaborations are also highly welcome!