Home

Awesome

AFter

Attention-based Fusion Router for RGBT Tracking. arXiv

<div align="center"> <img style="width:85%;" src="https://img2.imgtp.com/2024/05/01/Qv646jYC.png"/> </div>

Dataset

We use the LasHeR training set for training, GTOT, RGBT210, RGBT234, LasHeR testing set, VTUAVST for testing, and their project addresses are as follows:

Environment Preparation

Clone repo:

git clone https://github.com/Alexadlu/AFter.git
cd AFter

Our code is trained and tested with Python == 3.8, PyTorch == 1.8.1 and CUDA == 11.2 on NVIDIA GeForce RTX 4090, you may use a different version according to your GPU.

conda create -n after python=3.8.13
conda activate after
pip install -r requirements.txt

Training

  1. Modify the project path and dataset path in $PROJECT_ROOT$/ltr/admin/local.py.
  2. Download ToMP-50 pretrained weights and put it under $PROJECT_ROOT$/ltr/models/pretrained.
  3. Run the following command.
python ltr/run_training.py --train_module tomp --train_name tomp50_v1

Evaluation

  1. Modify the dataset path in $PROJECT_ROOT$/pytracking/evaluation/environment.py
  2. Put the checkpoint into $PROJECT_ROOT$/pytracking/networks and select the checkpoint name in $PROJECT_ROOT$/pytracking/parameter/tomp/tomp50.py. Or just modify the checkpoint path in $PROJECT_ROOT$/pytracking/parameter/tomp/tomp50.py.
  3. Run the following command.
python pytracking/run_tracker.py --tracker_name tomp --tracker_param tomp50 --runid 8600 --dataset_name lashertestingset

Results and Models

ModelGTOT(PR/SR)RGBT210(PR/SR)RGBT234(PR/SR)LasHeR(PR/NPR/SR)VTUAV(PR/SR)CheckpointRaw Result
AFter91.6 / 78.587.6 / 63.590.1 / 66.770.3 / 65.8 / 55.184.9 / 72.5downloaddownload

Acknowledgments

Our project is based on the pytracking framework and ToMP. Thanks for their contributions which help us to quickly implement our ideas.