Home

Awesome

TransCenter: Transformers with Dense Representations for Multiple-Object Tracking <br />

The work is accepted for TPAMI 2022.

An update towards a more efficient and powerful TransCenter, TransCenter-Lite!

The code for TransCenter and TransCenter-Lite is now available, you can find the code and pretrained models at https://gitlab.inria.fr/robotlearn/TransCenter_official.

TransCenter: Transformers with Dense Representations for Multiple-Object Tracking <br /> Yihong Xu, Yutong Ban, Guillaume Delorme, Chuang Gan, Daniela Rus, Xavier Alameda-Pineda <br /> [Paper] [Project]<br />

<div align="center"> <img src="https://github.com/yihongXU/TransCenter/raw/main/eTransCenter_pipeline.png" width="1200px" /> </div>

<br /><br /> MOT20 example: <br />

Bibtex

If you find this code useful, please star the project and consider citing: <br />

@misc{xu2021transcenter,
      title={TransCenter: Transformers with Dense Representations for Multiple-Object Tracking}, 
      author={Yihong Xu and Yutong Ban and Guillaume Delorme and Chuang Gan and Daniela Rus and Xavier Alameda-Pineda},
      year={2021},
      eprint={2103.15145},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

MOTChallenge Results

For TransCenter:

MOT17 public detections:

PretrainedMOTAMOTPIDF1FPFNIDS
CoCo71.9%80.5%64.1%27,356126,8604,118
CH75.9%81.2%65.9%30,190100,9994,626

MOT20 public detections:

PretrainedMOTAMOTPIDF1FPFNIDS
CoCo67.7%79.8%58.9%54,967108,3763,707
CH72.8%81.0%57.6%28,026110,3122,621

MOT17 private detections:

PretrainedMOTAMOTPIDF1FPFNIDS
CoCo72.7%80.3%64.0%33,807115,5424,719
CH76.2%81.1%65.5%40,10188,8275,394

MOT20 private detections:

PretrainedMOTAMOTPIDF1FPFNIDS
CoCo67.7%79.8%58.7%56,435107,1633,759
CH72.9%81.0%57.7%28,596108,9822,625

Note:

Acknowledgement

The code for TransCenterV2, TransCenter-Lite is modified and network pre-trained weights are obtained from the following repositories:

  1. The PVTv2 backbone pretrained models from PVTv2.
  2. The data format conversion code is modified from CenterTrack.

CenterTrack, Deformable-DETR, Tracktor.

@article{zhou2020tracking,
  title={Tracking Objects as Points},
  author={Zhou, Xingyi and Koltun, Vladlen and Kr{\"a}henb{\"u}hl, Philipp},
  journal={ECCV},
  year={2020}
}

@InProceedings{tracktor_2019_ICCV,
author = {Bergmann, Philipp and Meinhardt, Tim and Leal{-}Taix{\'{e}}, Laura},
title = {Tracking Without Bells and Whistles},
booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
month = {October},
year = {2019}}

@article{zhu2020deformable,
  title={Deformable DETR: Deformable Transformers for End-to-End Object Detection},
  author={Zhu, Xizhou and Su, Weijie and Lu, Lewei and Li, Bin and Wang, Xiaogang and Dai, Jifeng},
  journal={arXiv preprint arXiv:2010.04159},
  year={2020}
}

@article{zhang2021bytetrack,
  title={ByteTrack: Multi-Object Tracking by Associating Every Detection Box},
  author={Zhang, Yifu and Sun, Peize and Jiang, Yi and Yu, Dongdong and Yuan, Zehuan and Luo, Ping and Liu, Wenyu and Wang, Xinggang},
  journal={arXiv preprint arXiv:2110.06864},
  year={2021}
}

@article{wang2021pvtv2,
  title={Pvtv2: Improved baselines with pyramid vision transformer},
  author={Wang, Wenhai and Xie, Enze and Li, Xiang and Fan, Deng-Ping and Song, Kaitao and Liang, Ding and Lu, Tong and Luo, Ping and Shao, Ling},
  journal={Computational Visual Media},
  volume={8},
  number={3},
  pages={1--10},
  year={2022},
  publisher={Springer}
}

Several modules are from:

MOT Metrics in Python: py-motmetrics

Soft-NMS: Soft-NMS

DETR: DETR

DCNv2: DCNv2

PVTv2: PVTv2

ByteTrack: ByteTrack