Awesome
OpenDet
<img src="./docs/opendet2.png" width="78%"/>Expanding Low-Density Latent Regions for Open-Set Object Detection (CVPR2022)<br> Jiaming Han, Yuqiang Ren, Jian Ding, Xingjia Pan, Ke Yan, Gui-Song Xia.<br> arXiv preprint.
OpenDet2: OpenDet is implemented based on detectron2.
Setup
The code is based on detectron2 v0.5.
- Installation
Here is a from-scratch setup script.
conda create -n opendet2 python=3.8 -y
conda activate opendet2
conda install pytorch=1.8.1 torchvision cudatoolkit=10.1 -c pytorch -y
pip install detectron2==0.5 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/torch1.8/index.html
git clone https://github.com/csuhan/opendet2.git
cd opendet2
pip install -v -e .
- Prepare datasets
Please follow datasets/README.md for dataset preparation. Then we generate VOC-COCO datasets.
bash datasets/opendet2_utils/prepare_openset_voc_coco.sh
# using data splits provided by us.
cp datasets/voc_coco_ann datasets/voc_coco -rf
Model Zoo
We report the results on VOC and VOC-COCO-20, and provide pretrained models. Please refer to the corresponding log file for full results.
- Faster R-CNN
Method | backbone | mAP<sub>K↑</sub>(VOC) | WI<sub>↓</sub> | AOSE<sub>↓</sub> | mAP<sub>K↑</sub> | AP<sub>U↑</sub> | Download |
---|---|---|---|---|---|---|---|
FR-CNN | R-50 | 80.06 | 19.50 | 16518 | 58.36 | 0 | config model |
PROSER | R-50 | 79.42 | 20.44 | 14266 | 56.72 | 16.99 | config model |
ORE | R-50 | 79.80 | 18.18 | 12811 | 58.25 | 2.60 | config model |
DS | R-50 | 79.70 | 16.76 | 13062 | 58.46 | 8.75 | config model |
OpenDet | R-50 | 80.02 | 12.50 | 10758 | 58.64 | 14.38 | config model |
OpenDet | Swin-T | 83.29 | 10.76 | 9149 | 63.42 | 16.35 | config model |
- RetinaNet
Method | mAP<sub>K↑</sub>(VOC) | WI<sub>↓</sub> | AOSE<sub>↓</sub> | mAP<sub>K↑</sub> | AP<sub>U↑</sub> | Download |
---|---|---|---|---|---|---|
RetinaNet | 79.63 | 14.16 | 36531 | 57.32 | 0 | config model |
Open-RetinaNet | 79.64 | 10.74 | 17208 | 57.32 | 10.55 | config model |
Note:
- You can also download the pre-trained models at github release or BaiduYun with extracting code ABCD.
- The above results are reimplemented. Therefore, they are slightly different from our paper.
- The official code of ORE is at OWOD. So we do not plan to include ORE in our code.
Online Demo
Try our online demo at huggingface space.
Train and Test
- Testing
First, you need to download pretrained weights in the model zoo, e.g., OpenDet.
Then, run the following command:
python tools/train_net.py --num-gpus 8 --config-file configs/faster_rcnn_R_50_FPN_3x_opendet.yaml \
--eval-only MODEL.WEIGHTS output/faster_rcnn_R_50_FPN_3x_opendet/model_final.pth
- Training
The training process is the same as detectron2.
python tools/train_net.py --num-gpus 8 --config-file configs/faster_rcnn_R_50_FPN_3x_opendet.yaml
To train with the Swin-T backbone, please download swin_tiny_patch4_window7_224.pth and convert it to detectron2's format using tools/convert_swin_to_d2.py.
wget https://github.com/SwinTransformer/storage/releases/download/v1.0.0/swin_tiny_patch4_window7_224.pth
python tools/convert_swin_to_d2.py swin_tiny_patch4_window7_224.pth swin_tiny_patch4_window7_224_d2.pth
Citation
If you find our work useful for your research, please consider citing:
@InProceedings{han2022opendet,
title = {Expanding Low-Density Latent Regions for Open-Set Object Detection},
author = {Han, Jiaming and Ren, Yuqiang and Ding, Jian and Pan, Xingjia and Yan, Ke and Xia, Gui-Song},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2022}
}