Home

Awesome

TransT - Transformer Tracking [CVPR2021]

Official implementation of the TransT (CVPR2021) , including training code and trained models.

News

Tracker

TransT

[Paper] [Models(google)] [Models(baidu:iiau)] [Raw Results]

This work presents a attention-based feature fusion network, which effectively combines the template and search region features using attention. Specifically, the proposed method includes an ego-context augment module based on self-attention and a cross-feature augment module based on cross-attention. We present a Transformer tracking (named TransT) method based on the Siamese-like feature extraction backbone, the designed attention-based fusion mechanism, and the classification and regression head.

TransT is a very simple and efficient tracker, without online update module, using the same model and hyparameter for all test sets. TransT overview figure ECA and CFA

Results

For VOT2020, we add a mask branch to generate mask, without any hyparameter-tuning. The code of the mask branch will be released soon.

<table> <tr> <th>Model</th> <th>LaSOT<br>AUC (%)</th> <th>TrackingNet<br>AUC (%)</th> <th>GOT-10k<br>AO (%)</th> <th>VOT2020<br>EAO (%)</th> <th>TNL2K<br>AUC (%)</th> <th>OTB100<br>AUC (%)</th> <th>NFS<br>AUC (%)</th> <th>UAV123<br>AUC (%)</th> <th>Speed<br></th> <th>Params<br></th> </tr> <tr> <td>TransT-N2</td> <td>64.2</td> <td>80.9</td> <td>69.9</td> <td>-</td> <td>-</td> <td>68.1</td> <td>65.7</td> <td>67.0</td> <td>70fps</td> <td>16.7M</td> </tr> <tr> <td>TransT-N4</td> <td>64.9</td> <td>81.4</td> <td>72.3</td> <td>49.5</td> <td>51.0</td> <td>69.4</td> <td>65.7</td> <td>69.1</td> <td>50fps</td> <td>23.0M</td> </tr> </table>

Installation

This document contains detailed instructions for installing the necessary dependencied for TransT. The instructions have been tested on Ubuntu 18.04 system.

Install dependencies

You can modify these files to set the paths to datasets, results paths etc.

Quick Start

Traning

Evaluation

Getting Help

If you meet problem, please try searching our Github issues, if you can't find solutions, feel free to open a new issue.

Solution: You can just delete from pysot_toolkit.toolkit.utils.region import vot_overlap, vot_float2str in test.py if you don't test VOT2019/18/16. You can also build region by python setup.py build_ext --inplace in pysot_toolkit.

Citation

@inproceedings{TransT,
title={Transformer Tracking},
author={Chen, Xin and Yan, Bin and Zhu, Jiawen and Wang, Dong and Yang, Xiaoyun and Lu, Huchuan},
booktitle={CVPR},
year={2021}
}

Acknowledgement

This is a modified version of the python framework PyTracking based on Pytorch, also borrowing from PySOT and detr. We would like to thank their authors for providing great frameworks and toolkits.

Contact