Awesome
VA-Count
[ECCV 2024] Zero-shot Object Counting with Good Exemplars
[paper]
Zero-shot Object Counting with Good Exemplars
News🚀
- 2024.09.27: Our code is released.
- 2024.09.26: Our inference code has been updated, and the code for selecting exemplars and the training code will be coming soon.
- 2024.07.02: VA-Count is accepted by ECCV2024.
Overview
Overview of the proposed method. The proposed method focuses on two main elements: the Exemplar Enhancement Module (EEM) for improving exemplar quality through a patch selection integrated with Grounding DINO, and the Noise Suppression Module (NSM) that distinguishes between positive and negative class samples using density maps. It employs a Contrastive Loss function to refine the precision in identifying target class objects from others in an image.
Environment
pip install torch==1.10.0+cu111 torchvision==0.11.0+cu111 torchaudio==0.10.0 -f https://download.pytorch.org/whl/torch_stable.html
pip install timm==0.3.2
pip install numpy
pip install matplotlib tqdm
pip install tensorboard
pip install scipy
pip install imgaug
pip install opencv-python
pip3 install hub
For more information on Grounding DINO, please refer to the following link:
GroundingDINO We are very grateful for the Grounding DINO approach, which has been instrumental in our work!
Datasets
Preparing the datasets as follows:
./data/
|--FSC147
| |--images_384_VarV2
| | |--2.jpg
| | |--3.jpg
| |--gt_density_map_adaptive_384_VarV2
| | |--2.npy
| | |--3.npy
| |--annotation_FSC147_384.json
| |--Train_Test_Val_FSC_147.json
| |--ImageClasses_FSC147.txt
| |--train.txt
| |--test.txt
| |--val.txt
|--CARPK/
| |--Annotations/
| |--Images/
| |--ImageSets/
Inference
- For inference, you can download the model from Baidu-Disk, passward:paeh
python FSC_test.py --output_dir ./data/out/results_base --resume ./data/checkpoint_FSC.pth
Single and Multiple Object Classifier Training
python datasetmake.py
python biclassify.py
- You can also directly download the model from Baidu-Disk, passward:psum Save it in ./data/out/classify/
Generate exemplars
python grounding_pos.py --root_path ./data/FSC147/
python grounding_neg.py --root_path ./data/FSC147/
Train
CUDA_VISIBLE_DEVICES=0 python FSC_pretrain.py \
--epochs 500 \
--warmup_epochs 10 \
--blr 1.5e-4 --weight_decay 0.05
- You can also directly download the pre-train model from Baidu-Disk, passward:xynw Save it in ./data/
CUDA_VISIBLE_DEVICES=0 python FSC_train.py --epochs 1000 --batch_size 8 --lr 1e-5 --output_dir ./data/out/
Citation
@inproceedings{zhu2024zero,
title={Zero-shot Object Counting with Good Exemplars},
author={Zhu, Huilin and Yuan, Jingling and Yang, Zhengwei and Guo, Yu and Wang, Zheng and Zhong, Xian and He, Shengfeng},
booktitle={Proceedings of the European Conference on Computer Vision},
year={2024}
}
Acknowledgement
This project is based on the implementation from CounTR, we are very grateful for this work and GroundingDINO.