Home

Awesome

VadCLIP

This is the official Pytorch implementation of our paper: "VadCLIP: Adapting Vision-Language Models for Weakly Supervised Video Anomaly Detection" in AAAI 2024.

<a href="https://scholar.google.com.hk/citations?user=QkNqUH4AAAAJ" target="_blank">Peng Wu</a>, <a href="https://scholar.google.com/citations?user=ljzQLv4AAAAJ" target="_blank">Xuerong Zhou</a>, <a href="https://scholar.google.com.hk/citations?hl=zh-CN&user=1ZO7pHkAAAAJ" target="_blank">Guansong Pang</a>, <a href="https://paperswithcode.com/search?q=author%3ALingru+Zhou" target="_blank">Lingru Zhou</a>, <a href="https://scholar.google.com/citations?user=BSGy3foAAAAJ" target="_blank">Qingsen Yan</a>, <a href="https://scholar.google.com.au/citations?user=aPLp7pAAAAAJ" target="_blank">Peng Wang</a>, <a href="https://teacher.nwpu.edu.cn/m/en/1999000059.html" target="_blank">Yanning Zhang</a>

framework

Highlight

Training

Setup

We extract CLIP features for UCF-Crime and XD-Violence datasets, and release these features and pretrained models as follows:

BenchmarkCLIP[Baidu]CLIPModel[Baidu]Model
UCF-CrimeCode: 7yzpOneDriveCode: kq5uOneDrive
XD-ViolenceCode: v8twOneDriveCode: apw6OneDrive

The following files need to be adapted in order to run the code on your own machine:

Train and Test

After the setup, simply run the following command:

Traing and infer for XD-Violence dataset

python xd_train.py
python xd_test.py

Traing and infer for UCF-Crime dataset

python ucf_train.py
python ucf_test.py

References

We referenced the repos below for the code.

Citation

If you find this repo useful for your research, please consider citing our paper:

@article{wu2023vadclip,
  title={Vadclip: Adapting vision-language models for weakly supervised video anomaly detection},
  author={Wu, Peng and Zhou, Xuerong and Pang, Guansong and Zhou, Lingru and Yan, Qingsen and Wang, Peng and Zhang, Yanning},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence (AAAI)},
  year={2024}
}

@article{wu2023open,
  title={Open-Vocabulary Video Anomaly Detection},
  author={Wu, Peng and Zhou, Xuerong and Pang, Guansong and Sun, Yujia and Liu, Jing and Wang, Peng and Zhang, Yanning},
  journal={arXiv preprint arXiv:2311.07042},
  year={2023}
}