Awesome

ROMTrack

The official implementation of the ICCV 2023 paper Robust Object Modeling for Visual Tracking

[CVF Open Access] [Poster] [Video]

[Models and Raw Results] (Google Drive) [Models and Raw Results] (Baidu Netdisk: romt)

Base Models

Extended Models (Efficiency-Oriented)

Extended Models (Performance-Oriented)

:newspaper: News

[May 2, 2024]

• We release the extended models <u>ROMTrack-Large-384</u> for Performance-Oriented Visual Tracking!
• Models and Raw Results for all versions of ROMTrack are available on Google Drive or Baidu Netdisk.
• Code and script for VOT2020 evaluation are available now.

[April 18, 2024]

• We release the extended models <u>ROMTrack-Tiny-256</u> and <u>ROMTrack-Small-256</u> for Efficient Visual Tracking!
• We provide detailed information for all versions of ROMTrack, see Base Models and Extended Models above.

[April 17, 2024]

• Repository Upgrade is already done! Training and Evaluation using PyTorch 2.2.0 and Python 3.8 brings more efficiency.
• Training and Evaluation Devices for the upgraded code: RTX A6000, Intel(R) Xeon(R) Silver 4314 CPU @ 2.40GHz, Ubuntu 20.04.1 LTS.

[March 25, 2024]

• We upgrade the implementation to Python 3.8 and PyTorch 2.2.0!
• We update results on TNL2K!
• We update FPS metrics on RTX A6000 GPU for reference.

[March 21, 2024]

• We update 2 radar plots for visualization on LaSOT and LaSOT_ext.
• We post a blog on Zhihu, welcome for reading.

[October 18, 2023]

• We update paper in CVF Open Access version.
• We release poster and video.

[September 21, 2023]

• We release Models and Raw Results of ROMTrack.
• We refine README for more details.

[August 6, 2023]

• We release Code of ROMTrack.

[July 14, 2023]

• ROMTrack is accepted to ICCV2023!

:calendar: TODO

• Extended Models (Efficiency-Oriented & Performance-Oriented) for ROMTrack
• Repository Upgrade
• More Analysis (Radar Plot) and More Results (TNL2K Dataset)
• Code for ROMTrack
• Model Zoo and Raw Results
• Refine README

:star: Highlights

:rocket: New Tracking Framework pursing Robustness

• ROMTrack employes a robust object modeling design which can keep the inherent information of the target template and enables mutual feature matching between the target and the search region simultaneously.

• Robustness Comparison with SOTA methods (bounding box only) on VOT2020. <p align="center"> <img width="100%" src="asset/VOT2020.png" alt="VOT2020"/> </p>

:rocket: Strong Performance and Comparable Speed

• Performance on Benchmarks <p align="center"> <img width="90%" src="asset/Performance.png" alt="Performance"/> </p>
• Radar Analysis on LaSOT and LaSOT_ext <p align="center"> <img width="49%" src="asset/lasot_radar_plot_v2.png" alt="LaSOT_Radar"/> <img width="49%" src="asset/lasot_ext_radar_plot.png" alt="LaSOT_ext_Radar"/> </p>
• Speed, MACs, Params (Test on 1080Ti) <p align="center"> <img width="70%" src="asset/Speed.png" alt="Speed"/> </p>

:book: Install the environment

Use the Anaconda

conda create -n romtrack python=3.8
conda activate romtrack
bash install_pytorch.sh

:book: Data Preparation

Put the tracking datasets in ./data. It should look like:

${ROMTrack_ROOT}
 -- data
     -- lasot
         |-- airplane
         |-- basketball
         |-- bear
         ...
     -- lasot_ext
         |-- atv
         |-- badminton
         |-- cosplay
         ...
     -- got10k
         |-- test
         |-- train
         |-- val
     -- coco
         |-- annotations
         |-- train2017
     -- trackingnet
         |-- TRAIN_0
         |-- TRAIN_1
         ...
         |-- TRAIN_11
         |-- TEST

:book: Set project paths

Run the following command to set paths for this project

python tracking/create_default_local_file.py --workspace_dir . --data_dir ./data --save_dir .

After running this command, you can also modify paths by editing these two files

lib/train/admin/local.py  # paths about training
lib/test/evaluation/local.py  # paths about testing

:book: Train ROMTrack

Training with multiple GPUs using DDP. More details of other training settings can be found at tracking/train_romtrack.sh

bash tracking/train_romtrack.sh

:book: Test and evaluate ROMTrack on benchmarks

• LaSOT/LaSOT_ext/GOT10k-test/TrackingNet/OTB100/UAV123/NFS30.
  • More details of test settings can be found at tracking/test_romtrack.sh

bash tracking/test_romtrack.sh

• VOT2020. Current version is vot-toolkit(==0.5.3) and vot-trax(==3.0.3).
  • Take ROMTrack-Large-384 below as an example.

### Evaluate ROMTrack-Large-384 with AlphaRefine
vot evaluate --workspace ./external/vot2020/ROMTrack_large_384 ROMTrack_large_384_AR
vot analysis --nocache --workspace ./external/vot2020/ROMTrack_large_384 ROMTrack_large_384_AR

### Evaluate ROMTrack-Large-384 without AlphaRefine
vot evaluate --workspace ./external/vot2020/ROMTrack_large_384 ROMTrack_large_384
vot analysis --nocache --workspace ./external/vot2020/ROMTrack_large_384 ROMTrack_large_384

:book: Compute FLOPs/Params and test speed

bash tracking/profile_romtrack.sh

:book: Visualization

We provide attention maps and feature maps for several sequences on LaSOT. Detailed analysis can be found in our paper.

:bookmark: Acknowledgments

• Thanks for STARK, PyTracking and MixFormer Library, which helps us to quickly implement our ideas and test our performances.
• Our implementation of the ViT is modified from the Timm repo.

:pencil: Citation

If our work is useful for your research, please feel free to star :star: and cite our paper:

@InProceedings{Cai_2023_ICCV,
    author    = {Cai, Yidong and Liu, Jie and Tang, Jie and Wu, Gangshan},
    title     = {Robust Object Modeling for Visual Tracking},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2023},
    pages     = {9589-9600}
}