Home

Awesome

Interleaving One-Class and Weakly-Supervised Models with Adaptive Thresholding for Unsupervised Video Anomaly Detection

The official PyTorch implementation of the paper "Interleaving One-Class and Weakly-Supervised Models with Adaptive Thresholding for Unsupervised Video Anomaly Detection".

Framework_Overview

Getting Started

This code was tested on Ubuntu 18.04.6 LTS and requires:

Setup Conda Environment:

git clone https://github.com/benedictstar/Joint-VAD
cd Joint-VAD

# Conda environment setup
conda env create -f environment.yml
conda activate Joint

Data Preparation

Since our work combines two open-source models STG-NF and RTFM, you should organize the data following the guidance of each original repository.

The following files need to be adapted in order to run the code on your own machine:

Data Folder Structure

.
├── weakly_ShanghaiTech
│   ├── gt
│   │   └── test_frame_mask
│   ├── list
│   │   ├── gt-sh.npy
│   │   ├── make_gt_sh.py
│   │   ├── make_list_sh.py
│   │   ├── shanghai-i3d-test-10crop.list
│   │   ├── shanghai-i3d-train-10crop.list
│   │   └── test_frame_mask
│   ├── pose
│   │   ├── test
│   │   └── train
│   ├── shanghai-num.npy
│   └── SH_Train.txt
└── weakly_UBnormal
    ├── gt
    │   └── test_gt
    ├── list
    │   ├── gt-ub.npy
    │   ├── make_gt_ub.py
    │   ├── make_list_ub.py
    │   ├── ubnormal-i3d-test-10crop.list
    │   ├── ubnormal-i3d-train-10crop.list
    │   └── ubnormal-i3d-validation-10crop.list
    ├── pose
    │   ├── abnormal_train
    │   ├── test
    │   ├── train
    │   └── validation
    ├── ub-num.npy
    └── UB_Train.txt

Training/Testing

Training and Evaluating is run using:

python main.py --dataset [ShanghaiTech\UBnormal] --ab_ratio [0.15\0.3] --seg_len [24\16]

Evaluation of our pretrained model can be done using:

STG-NF Model for ShanghaiTech:

python OCC_eval.py --dataset ShanghaiTech --Occ_checkpoint pretrained_models/SH_Occ_best_auc_in_UVAD_82.57.pth.tar

RTFM Model for ShanghaiTech:

python WS_eval.py --dataset ShanghaiTech --Weakly_pretrained_ckpt pretrained_models/SH_WS_best_auc_in_UVAD_88.52.pkl

STG-NF Model for UBnormal:

python OCC_eval.py --dataset UBnormal --seg_len 16 --Occ_checkpoint pretrained_models/UB_Occ_best_auc_in_UVAD_74.82.pth.tar

RTFM Model for UBnormal:

python WS_eval.py --dataset UBnormal --Weakly_pretrained_ckpt pretrained_models/UB_WS_best_auc_in_UVAD_63.26.pkl

Acknowledgments

Our code is based on code from: