Awesome

TubeTK

TubeTK is an one-step end-to-end multi-object tracking method, which is the first end-to-end open-source system that achieves 60+ MOTA on MOT-16 (64 MOTA) and MOT-17 (63 MOTA) datasets. Our paper "TubeTK: Adopting Tubes to Track Multi-Object in a One-Step Training Model" is accepted as an oral paper on CVPR-2020.

Contents

• TubeTK
• Contents
• Results
  • MOT-16
  • MOT-17
• Installation
• Quick Start
  • Demo
  • Evaluation
  • Train

Results

MOT-16

Results on MOT-16 dataset:

MOT-17

Results on MOT-17 dataset:

Installation

1. Get the code and build related modules:

     git clone ...(TO BE CONFIRM)
     cd TubeTK/install
     ./compile.sh
     # if something wrong, try:
     # sudo ldconfig <path/to/cuda>/lib64
     cd ..
   

2. Install pytorch 1.10 and other dependencies:

pip install -r requirements.txt

3. If the memory of your GPU < 16G, then you need NVIDIA APEX to conduct the mixed precision training.

   1. Install Apex:

   git clone https://github.com/NVIDIA/apex
   cd apex
   pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./
   # if something wrong with the above pip install, try:
   # pip install -v --no-cache-dir ./
   

   2. We provide the --apex option to train with the APEX, see Quick Start for detail.

4. Run fetch_model.sh to download our pre-trained models. Or download the models manually and put them in ./models:

   1. 3DResNet50_original (Baidu pan | Google drive)
   <!---2. 3DResNet50_small ([Baidu pan]() | [Google drive]())-->

Quick Start

Demo

Run TubeTK for a video and visualization the results with:

python launch.py --nproc_per <num of GPU> --training_script demo.py --batch_size=3 --config configs/TubeTK_resnet_50_FPN_8frame_1stride.yaml --video_url <folder/to/the/videos> --output_dir ./vis_video

Evaluation on MOT-17 (16)

1. Download the data from MOT Challenge, and put or link it to ./data

2. To get the tracking result with:

   python launch.py --nproc_per <num of GPU> --training_script evaluate.py --batch_size 3 --config configs/TubeTK_resnet_50_FPN_8frame_1stride.yaml --trainOrTest test
   

3. To get the visualization with:

   python Visualization/Vis_Res.py --mode test
   

   The visualization videos are stored in ./vis_video .

Train on MOT-17 (16)

1. Download the data from MOT Challenge, and put or link it to ./data

2. Get the ground truth Btubes with:

   python ./pre_processing/get_tubes_MOT17.py
   

3. Train the model with:

   python launch.py --nproc_per <num of GPU> --training_script main.py --batch_size 1 --config ./configs/TubeTK_resnet_50_FPN_8frame_1stride.yaml
   

   If out of memory, try:

   python launch.py --nproc_per <num of GPU> --training_script main.py --batch_size 1 --config ./configs/TubeTK_resnet_50_FPN_8frame_1stride.yaml --apex
   

   If still out of memory, modify the configuration file: TubeTK_resnet_50_FPN_8frame_1stride.yaml:

   tube_limit: 500  # or 300
   

Citation

@inproceedings{pang2020tubetk,
   title={TubeTK: Adopting Tubes to Track Multi-Object in a One-Step Training Model},
   author={Pang, Bo and Li, Yizhuo and Zhang, Yifan and Li, Muchen and Lu, Cewu},
   booktitle={CVPR},
   year={2020}
}

License

TubeTK is freely available for free non-commercial use, and may be redistributed under these conditions.