Awesome

NeighborTrack

Update: Add model speed and some experiment in CVPRW:

NeighborTrack: Single Object Tracking by Bipartite Matching With Neighbor Tracklets and Its Applications to Sports

Old version:

NeighborTrack: Improving Single Object Tracking by Bipartite Matching with Neighbor Tracklets

This paper was accepted by the 9th International Workshop on Computer Vision in Sports (CVsports) 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPR)

Single Object Tracking(SOT) post-processing method by using cycle consistency and neighbor(python version)

Some SOT model codes are from OSTrack, votchallenge, Ocean, TransT, pytracking, and Mixformer. Thanks to these projects a lot.

Website: OSTrack, TransT, Votchallenge, Ocean, pytracking, Mixformer,

KalmanFilter implement

SoftNMS implement

Demo videos

Results

Models and source results link

LaSOT,GOT10K,TrackingNet,UAV123,OTB100 (baseline from OSTrack github code)

Note: UAV123 has some long-term tracking videos, and it needs more temporal information, if use standard setting tau=9, it cannot improve AUC, we set tau=27 on the whole dataset

Votchallenge

Bibtex

@InProceedings{Chen_2023_CVPR,
    author    = {Chen, Yu-Hsi and Wang, Chien-Yao and Yang, Cheng-Yun and Chang, Hung-Shuo and Lin, Youn-Long and Chuang, Yung-Yu and Liao, Hong-Yuan Mark},
    title     = {NeighborTrack: Single Object Tracking by Bipartite Matching With Neighbor Tracklets and Its Applications to Sports},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
    month     = {June},
    year      = {2023},
    pages     = {5138-5147}
}

Quick start

1. Install Environment

my driver version: NVIDIA-SMI 465.19.01 Driver Version: 465.19.01 Python 3.7.7 (default, Mar 23, 2020, 22:36:06) torch.version.cuda=10.1

pip install munkres==1.1.4
pip install shapely

Other environments depend on your base model, e.g. OSTrack:

Example of my Environment please see This file.

cd trackers/ostrack
sh example_ostrack_install.sh

2. Download the dataset and models, then put them on each path

Models and source results link

More information for model paths

Get results from NeighborTrack with OSTrack

Work space is in 'NeighborTrack/trackers/ostrack/', please remember to change the dataset and model's root.

More information :OSTrack user's guide

LaSOT, GOT-10K

cd /your_path/trackers/ostrack/
sh test.sh
#or
#lasot example
python tracking/test.py ostrack vitb_384_mae_ce_32x4_ep300_neighbor --dataset lasot --threads 24 --num_gpus 8 --neighbor 1
#python tracking/analysis_results.py 

#got-10K example
python tracking/test.py ostrack vitb_384_mae_ce_32x4_ep300_neighbor --dataset got10k_test --threads 16 --num_gpus 8 --neighbor 1 
#to use got-10K train_from_got10K_only
python tracking/test.py ostrack vitb_384_mae_ce_32x4_got10k_ep100_neighbor --dataset got10k_test --threads 16 --num_gpus 8 --neighbor 1 

VOT challenge

vot test ostrackNeighbor
vot test ostrackNeighborAR
vot evaluate --workspace ./vot2022st ostrackNeighbor
vot analysis --workspace vot2022st ostrackNeighbor

vot evaluate --workspace ./vot2021 ostrackNeighborAR
vot analysis --workspace vot2021 ostrackNeighborAR

setting vot workspace example VOT trackers example:trackers.ini, ostrack_384_vot_neighbor.py

If you want to know how to create workspace of vot2022st vot2020 vot2021 dataset, please seen Votchallenge:

In your own video

sh video_test.sh
# or
python tracking/video_demo_neighbor.py ostrack vitb_384_mae_ce_32x4_ep300_neighbor ./cup1.avi  \
   --optional_box 1109 531 82 135 --save_results --debug 1 --save_img
#optional_box is GT in first frame.

How to use NeighborTrack in your own SOT tracker:

1. Create dependent functions:

There is a simple code from the votchallenge NCC tracker, add 3 functions to use our method(initialize, track_neighbor, and update_center). Please see: https://github.com/franktpmvu/NeighborTrack/blob/c889695427a2288b42e31cd0f9e0f7e509244729/trackers/example_ncc_tracker.py#L14
After adding functions are seems like: https://github.com/franktpmvu/NeighborTrack/blob/c889695427a2288b42e31cd0f9e0f7e509244729/trackers/example_ncc_tracker.py#L51

Remember, the tracker should have 2 independent models forward/reverse because all of the SOT methods will forget the tracking target after initialization, if just 1 forward/backward tracker, it cannot switch forward/backward mission and ensure the forward answer doesn't have any change (even didn't use our method to change output, just use the same tracker to track any other object, your forward output will not come back to original answer, because the memory of tracker is changed.)

Other example: ostrack add 3 functions https://github.com/franktpmvu/NeighborTrack/blob/c889695427a2288b42e31cd0f9e0f7e509244729/trackers/ostrack/lib/test/evaluation/tracker.py#L328

More details:

2.Usage:

Init

https://github.com/franktpmvu/NeighborTrack/blob/c889695427a2288b42e31cd0f9e0f7e509244729/trackers/ostrack/tracking/ostrack_384_vot_neighbor.py#L63

Get tracking answer

https://github.com/franktpmvu/NeighborTrack/blob/c889695427a2288b42e31cd0f9e0f7e509244729/trackers/ostrack/tracking/ostrack_384_vot_neighbor.py#L70

Tracker and invtracker are original ostrack, you can change them by your SOT tracker.

region = [x,y,w,h],(x y = top left)

image = image by your model input, for example ostrack's image = numpy.array(img[h,w,3(RGB)])

No module named xxxx

If you see this error, please add 3 paths on tracking/test.py

{}\NeighborTrack\trackers\ostrack\lib\test\tracker

{}\NeighborTrack\trackers\ostrack\tracking

{}   (= NeighborTrack\..\)