Awesome

Multiple Object Tracking as ID Prediction

This is the official PyTorch implementation of our paper:

Multiple Object Tracking as ID Prediction <br> :mortar_board: Ruopeng Gao, Yijun Zhang, Limin Wang <br> :e-mail: Primary contact: ruopenggao@gmail.com

<br> <br> <br>

:mag: Overview

TL; DR. MOTIP proposes a new perspective to regard the multi-object tracking task as an ID prediction problem. It directly predicts the ID labels for each object in the tracking process, which is more straightforward and effective.

Abstract. In Multiple Object Tracking (MOT), tracking-by-detection methods have stood the test for a long time, which split the process into two parts according to the definition: object detection and association. They leverage robust single-frame detectors and treat object association as a post-processing step through hand-crafted heuristic algorithms and surrogate tasks. However, the nature of heuristic techniques prevents end-to-end exploitation of training data, leading to increasingly cumbersome and challenging manual modification while facing complicated or novel scenarios. In this paper, we regard this object association task as an End-to-End in-context ID prediction problem and propose a streamlined baseline called MOTIP. Specifically, we form the target embeddings into historical trajectory information while considering the corresponding IDs as in-context prompts, then directly predict the ID labels for the objects in the current frame. Thanks to this end-to-end process, MOTIP can learn tracking capabilities straight from training data, freeing itself from burdensome hand-crafted algorithms. Without bells and whistles, our method achieves impressive state-of-the-art performance in complex scenarios like DanceTrack and SportsMOT, and it performs competitively with other transformer-based methods on MOT17. We believe that MOTIP demonstrates remarkable potential and can serve as a starting point for future research.

:fire: News

• <span style="font-variant-numeric: tabular-nums;">2024.05.06</span>: We release the training code and scripts :hugs:. The pre-training scripts will be released later :soon:. Now you can directly download pre-trained weights from the Cloud :cloud:.

• <span style="font-variant-numeric: tabular-nums;">2024.03.28</span>: We release the inference code, you can evaluate the model following the instructions :tada:. Our model weights and logs are available in the Google Drive :cloud:.

• <span style="font-variant-numeric: tabular-nums;">2024.03.26</span>: The paper is released on arXiv, the code will be available in several days. Welcome to watch our repository for the latest updates :pushpin:.

:chart_with_upwards_trend: ​Main Results

:dancer: ​DanceTrack

:basketball: ​SportsMOT

:walking: ​MOT17

:dash: Quick Start

• See INSTALL.md for instructions of installing required components.
• See DATASET.md for datasets download and preparation.
• See PRETRAIN.md for how to get pretrained DETR weights.
• See GET_STARTED.md for how to get started with our MOTIP.
• See MISCELLANEOUS.md for other miscellaneous settings unrelated to the model structure, such as logging.

:bouquet: Acknowledgements

This project is built upon Deformable DETR, MOTR, DAB-Deformable DETR, TrackEval. Thanks to the contributors of these great codebases.

:pencil2: Citation

If you think this project is helpful, please feel free to leave a :star: and cite our paper:

@article{MOTIP,
  title={Multiple Object Tracking as ID Prediction},
  author={Gao, Ruopeng and Zhang, Yijun and Wang, Limin},
  journal={arXiv preprint arXiv:2403.16848},
  year={2024}
}

:star2: Stars