Home

Awesome

ZBS: Zero-shot Background Subtraction via Instance-level Background Modeling and Foreground Selection

Introduction

This repository is an official implementation of the ZBS. ZBS fully utilizes the advantages of zero-shot object detection to build the open-vocabulary instance-level background model. It can detect most of the categories in the real world and can detect the unseen foreground categories outside the pre-defined categories. ZBS achieves remarkably 4.70% F-Measure improvements over state-of-the-art unsupervised methods.

<p align="center"> <img src='docs/arch.svg' align="center" height="600px"> </p>

Features

Instructions

See GET_STARTED.md.

Main Results

Overall and per-category F-Measure comparison of different Unsupervised BGS methods on the CDnet 2014 dataset.

Unsupervised BGSbaselinecamjittdynbgintmotshadowthermalbadwealowfrnightPTZturbulOverall
PAWCS0.93970.81370.89380.77640.89130.83240.81520.65880.41520.46150.64500.7403
SuBSENSE0.95030.81520.81770.65690.89860.81710.86190.64450.55990.34760.77920.7408
WisenetMD0.94870.82280.83760.72640.89840.81520.86160.64040.57010.33670.83040.7535
SWCD0.92140.74110.86450.70920.87790.85810.82330.73740.58070.45450.77350.7583
SemanticBGS0.96040.83880.94890.78780.94780.82190.82600.78880.50140.56730.69210.7892
RTSS0.95970.83960.93250.78640.95510.85100.86620.67710.52950.54890.76300.7917
RT-SBS-v20.95350.82330.92170.89460.94970.86970.82790.73410.56290.58080.73150.8045
ZBS (Ours)0.96530.95450.92900.87580.97650.86980.92290.74330.68000.81330.63580.8515

Overall and per-category result of ZBS on the CDnet 2014 dataset.

CategoryRecallSpecificityPWCPrecisionF-Measure
badWea0.90490.99880.27550.94390.9229
baseline0.97090.99880.22370.96030.9653
camjitt0.95430.99790.40220.95540.9545
dynbg0.92690.99960.09510.93400.9290
intmot0.82540.99651.68640.94810.8758
lowfr0.73020.99880.32790.75840.7433
night0.63410.99581.24770.76660.6800
PTZ0.74900.99970.23870.92230.8133
shadow0.97120.99910.20970.98190.9765
thermal0.84750.99541.16860.90400.8698
turbul0.72860.99840.31980.60750.6358
Overall0.84030.99810.56320.88020.8515

Citation

If you find this project useful for your research, please consider citing this paper.

@inproceedings{
an2023zbs,
title={{ZBS}: Zero-shot Background Subtraction via instance-level background modeling and foreground selection},
author={Yongqi An and Xu Zhao and Tao Yu and Haiyun Guo and Chaoyang Zhao and Ming Tang and Jinqiao Wang},
booktitle={Conference on Computer Vision and Pattern Recognition 2023},
year={2023},
url={https://openreview.net/forum?id=f-9UZN4GEV}
}

Acknowledgement

Our repository is mainly built upon Detic.