Home

Awesome

News: I've released my new work on MOT. This tracker is named as Deep Motion Modeling Network (DMMN) which can simultaneously perform detection, feature modeling, association, and even prediction. Besides, this work is based on the framework of SSD, which can be extended to the framework of Faster RCNN, YOLO v3/4.

DAN (Deep Affinity Network)

Table Of Content

Purpose

DAN is an end-to-end deep learning network during train phase, whose purpose is to extract the predetected objects' feature and performs pairing permutations of those features in any two frames to infer object affinities. Besides, DAN also accounts for multiple objects appearing and disappearing between video frames.

Note: The repository was built with the name "SST". For the brevity and easy understanding, we change the repository name by "DAN".

Deep Affinity Network (DAN)

The network can be divided into two parts: feature extractor and affinity extractor. The feature extractor extracts each detected objects' features. The affinity extractor leverage those features to compute object data association matrix.

Deep tracking with DAN deployment. The DAN extracts the feature of each detected object provided by the Detector, and also estimate the affinity matrices.

The input & output of network

Training phase

NameItems
Input- Two video frames (with any interval frame)<br/>- Center of the pre-detected boxes <br>- Binary data association matrix
Output- Loss

Testing phase

NameItems
Input- Two video frames (with any interval frame)<br/>- Center of the pre-detected boxes
Output- Features of pre-detected boxes <br/>- Predicted matrices used to compute object affinities

Task

Current Task

TitleStartDueDetailStatus
Update the result of DPM2018/10/31-Update the result of DPM of MOT17Continuing

History Task

TitleStartDueDetailStatus
Update ReadMe2018/10/282018/10/31Update the README according paperFinish
Evaluate MOT152018/09/152018/10/20Evaluate on MOT15Finish
Re-evaluate MOT172018/08/102018/09/01Re-evaluate on MOT17Finish
Fix Result of UA-DETRAC2018/08/012018/09/13Fix the result of UA-DETRACFinish
Start UA-DETRAC2018/04/232018/09/13Evaluate on UA-DETRACFinish
KITTI2018/04/112018/04/23Training, OptimizeGive up:(
Re-Train KITTI2018/04/182018/04/20with gap frame 5Finish
Continue Train KITTI2018/04/162018/04/18Continue training KITTIFinish
Training KITTI dataset2018/04/112018/04/16Training KITTI datasetFinish
Evaluate On MOT172018/022018/03/28Top 1 at MOT17 :)Finish
Design Framework2017/11/152018/01/14The tracking frameworkFinish
Select Dataset2017/11/102017/11/15MOT17, KITTI, UA-DETRACFinish
Designing network2017/10/152017/11/15Designing the network for trainingFinish
Start the project2017/10/012017/10/15Start the idea based on SSDFinish

Requirement

The requirement as follows:

NameVersion
cuda8.0
python3.5
pytorch0.3.1

Before going on, we recommend minconda. After you install miniconda, then create a new environment named DAN, then run the following script:

conda create -n DAN python=3.5
source activate DAN

Run the following script to install the required python packages:

If you're in China, you'd better run the following script in order to speed up the downloading.

pip install pip -U
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
cd SST
pip install -r requirement.txt

SST is the path of our repository folder.

Dataset

Our method can be evaluated on MOT17, MOT15 and UA-DETRAC.

MOT15 and MOT17 is for pedestrian tracking.

UA-DETRAC focuses is for vehicle tracking.

Train & Test On MOT17

For the simplity, we only introduce how to train and evaluate on MOT17

Download dataset

  1. Download the mot 17 dataset 5.5 GB and development kit 0.5 MB.

  2. Unzip this the dataset. Here is my file structure.

    MOT17
    ├── test
    └── train
    

Test

  1. Download the weigths from Dropbox or BaiYunPan to the SST/weights folder

  2. Modify SST/config/config.py as follows:

    # You need to modify line 8, 72, 73 and 74.
    8	current_select_configure = 'init_test_mot17' # need use 'init_test_mot17'
    ...	...
    70	def init_test_mot17():
    71        config['resume'] = './weights/sst300_0712_83000.pth'
    72        config['mot_root'] = 'replace by your dataset folder' 
    73		  config['save_folder'] = 'replace by your save folder'
    74        config['log_folder'] = 'replace by your log folder'
    75        config['batch_size'] = 1
    76        config['write_file'] = True
    77        config['tensorboard'] = True
    78        config['save_combine'] = False
    79        config['type'] = 'test' # or 'train'
    
  3. run test_mot17.py

    cd <SST>
    python test_mot17.py
    

    The result is shown as follows

    The title of each detected boxes represents (track id, detection id)

Train

  1. Download the vgg weights from Dropbox or BaiYunPan to the weights folder

  2. Modify SST/config/config.py as follows:

    # you need to modify line 8, 87, 89, 90 and 91.
    8	current_select_configure = 'init_train_mot17' # need use 'init_train_mot17'
    ...	...
    85	def init_train_mot17():
    86		config['epoch_size'] = 664
    87		config['mot_root'] = 'replace by your mot17 dataset folder'
    88		config['base_net_folder'] = './weights/vgg16_reducedfc.pth'
    89		config['log_folder'] = 'replace by your log folder'
    90		config['save_folder'] = 'replace by your save folder'
    91		config['save_images_folder'] = 'replace by your image save folder'
    92		config['type'] = 'train'
    93		config['resume'] = None # None means training from sketch.
    94		config['detector'] = 'DPM'
    95		config['start_iter'] = 0
    96		config['iteration_epoch_num'] = 120
    97		config['iterations'] = config['start_iter'] + config['epoch_size'] *     config['iteration_epoch_num'] + 50
    98		config['batch_size'] = 4
    99		config['learning_rate'] = 1e-2
    100		config['learning_rate_decay_by_epoch'] = (50, 80, 100, 110)
    101		config['save_weight_every_epoch_num'] = 5
    102		config['min_gap_frame'] = 0
    103		config['max_gap_frame'] = 30
    104		config['false_constant'] = 10
    105		config['num_workers'] = 16
    106		config['cuda'] = True
    107		config['max_object'] = 80
    108		config['min_visibility'] = 0.3
    
  3. Run train_mot17.py

    cd <SST>
    python train_mot17.py
    

Citation

If you use this source code or part of the source code. It is necessary to cite the following paper:

Sun. S., Akhtar, N., Song, H., Mian A., & Shah M. (2018). Deep Affinity Network for Multiple Object Tracking, Retrieved from https://arxiv.org/abs/1810.11780

Acknowledge

This code is based on ssd.pytorch

License

The methods provided on this page are published under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License . This means that you must attribute the work in the manner specified by the authors, you may not use this work for commercial purposes and if you alter, transform, or build upon this work, you may distribute the resulting work only under the same license. If you are interested in commercial usage you can contact us for further options.

Cool Examples