Home

Awesome

SiamMask

NEW: now including code for both training and inference!

PWC

This is the official implementation with training code for SiamMask (CVPR2019). For technical details, please refer to:

Fast Online Object Tracking and Segmentation: A Unifying Approach <br /> Qiang Wang*, Li Zhang*, Luca Bertinetto*, Weiming Hu, Philip H.S. Torr (* denotes equal contribution) <br /> CVPR 2019 <br /> [Paper] [Video] [Project Page] <br />

<div align="center"> <img src="http://www.robots.ox.ac.uk/~qwang/SiamMask/img/SiamMask.jpg" width="600px" /> </div>

Bibtex

If you find this code useful, please consider citing:

@inproceedings{wang2019fast,
    title={Fast online object tracking and segmentation: A unifying approach},
    author={Wang, Qiang and Zhang, Li and Bertinetto, Luca and Hu, Weiming and Torr, Philip HS},
    booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
    year={2019}
}

Contents

  1. Environment Setup
  2. Demo
  3. Testing Models
  4. Training Models

Environment setup

This code has been tested on Ubuntu 16.04, Python 3.6, Pytorch 0.4.1, CUDA 9.2, RTX 2080 GPUs

git clone https://github.com/foolwood/SiamMask.git && cd SiamMask
export SiamMask=$PWD
conda create -n siammask python=3.6
source activate siammask
pip install -r requirements.txt
bash make.sh
export PYTHONPATH=$PWD:$PYTHONPATH

Demo

cd $SiamMask/experiments/siammask_sharp
wget http://www.robots.ox.ac.uk/~qwang/SiamMask_VOT.pth
wget http://www.robots.ox.ac.uk/~qwang/SiamMask_DAVIS.pth
cd $SiamMask/experiments/siammask_sharp
export PYTHONPATH=$PWD:$PYTHONPATH
python ../../tools/demo.py --resume SiamMask_DAVIS.pth --config config_davis.json
<div align="center"> <img src="http://www.robots.ox.ac.uk/~qwang/SiamMask/img/SiamMask_demo.gif" width="500px" /> </div>

Testing

cd $SiamMask/data
sudo apt-get install jq
bash get_test_data.sh
cd $SiamMask/experiments/siammask_sharp
wget http://www.robots.ox.ac.uk/~qwang/SiamMask_VOT.pth
wget http://www.robots.ox.ac.uk/~qwang/SiamMask_VOT_LD.pth
wget http://www.robots.ox.ac.uk/~qwang/SiamMask_DAVIS.pth
bash test_mask_refine.sh config_vot.json SiamMask_VOT.pth VOT2016 0
bash test_mask_refine.sh config_vot.json SiamMask_VOT.pth VOT2018 0
bash test_mask_refine.sh config_vot.json SiamMask_VOT.pth VOT2019 0
bash test_mask_refine.sh config_vot18.json SiamMask_VOT_LD.pth VOT2016 0
bash test_mask_refine.sh config_vot18.json SiamMask_VOT_LD.pth VOT2018 0
python ../../tools/eval.py --dataset VOT2016 --tracker_prefix C --result_dir ./test/VOT2016
python ../../tools/eval.py --dataset VOT2018 --tracker_prefix C --result_dir ./test/VOT2018
python ../../tools/eval.py --dataset VOT2019 --tracker_prefix C --result_dir ./test/VOT2019
bash test_mask_refine.sh config_davis.json SiamMask_DAVIS.pth DAVIS2016 0
bash test_mask_refine.sh config_davis.json SiamMask_DAVIS.pth DAVIS2017 0
bash test_mask_refine.sh config_davis.json SiamMask_DAVIS.pth ytb_vos 0

Results

These are the reproduction results from this repository. All results can be downloaded from our project page.

<sub>Tracker</sub><sub>VOT2016</br>EAO / A / R</sub><sub>VOT2018</br>EAO / A / R</sub><sub>DAVIS2016</br>J / F</sub><sub>DAVIS2017</br>J / F</sub><sub>Youtube-VOS</br>J_s / J_u / F_s / F_u</sub><sub>Speed</sub>
<sub>SiamMask-box</sub><sub>0.412/0.623/0.233</sub><sub>0.363/0.584/0.300</sub>- / -- / -- / - / - / -<sub>77 FPS</sub>
<sub>SiamMask</sub><sub>0.433/0.639/0.214</sub><sub>0.380/0.609/0.276</sub><sub>0.713/0.674</sub><sub>0.543/0.585</sub><sub>0.602/0.451/0.582/0.477</sub><sub>56 FPS</sub>
<sub>SiamMask-LD</sub><sub>0.455/0.634/0.219</sub><sub>0.423/0.615/0.248</sub>- / -- / -- / - / - / -<sub>56 FPS</sub>

Note:

Training

Training Data

Download the pre-trained model (174 MB)

(This model was trained on the ImageNet-1k Dataset)

cd $SiamMask/experiments
wget http://www.robots.ox.ac.uk/~qwang/resnet.model
ls | grep siam | xargs -I {} cp resnet.model {}

Training SiamMask base model

cd $SiamMask/experiments/siammask_base/
bash run.sh
bash test_all.sh -s 1 -e 20 -d VOT2018 -g 4  # test all snapshots with 4 GPUs
#bash test_all.sh -m [best_test_model] -d VOT2018 -n [thread_num] -g [gpu_num] # 8 threads with 4 GPUS
bash test_all.sh -m snapshot/checkpoint_e12.pth -d VOT2018 -n 8 -g 4 # 8 threads with 4 GPUS

Training SiamMask model with the Refine module

cd $SiamMask/experiments/siammask_sharp
bash run.sh <best_base_model>
bash run.sh checkpoint_e12.pth
bash test_all.sh -s 1 -e 20 -d VOT2018 -g 4

Training SiamRPN++ model (unofficial)

cd $SiamMask/experiments/siamrpn_resnet
bash run.sh
bash test_all.sh -h
bash test_all.sh -s 1 -e 20 -d VOT2018 -g 4

License

Licensed under an MIT license.