Home

Awesome

OpenDet

<img src="./docs/opendet2.png" width="78%"/>

Expanding Low-Density Latent Regions for Open-Set Object Detection (CVPR2022)<br> Jiaming Han, Yuqiang Ren, Jian Ding, Xingjia Pan, Ke Yan, Gui-Song Xia.<br> arXiv preprint.

OpenDet2: OpenDet is implemented based on detectron2.

Setup

The code is based on detectron2 v0.5.

Here is a from-scratch setup script.

conda create -n opendet2 python=3.8 -y
conda activate opendet2

conda install pytorch=1.8.1 torchvision cudatoolkit=10.1 -c pytorch -y
pip install detectron2==0.5 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/torch1.8/index.html
git clone https://github.com/csuhan/opendet2.git
cd opendet2
pip install -v -e .

Please follow datasets/README.md for dataset preparation. Then we generate VOC-COCO datasets.

bash datasets/opendet2_utils/prepare_openset_voc_coco.sh
# using data splits provided by us.
cp datasets/voc_coco_ann datasets/voc_coco -rf

Model Zoo

We report the results on VOC and VOC-COCO-20, and provide pretrained models. Please refer to the corresponding log file for full results.

MethodbackbonemAP<sub>K↑</sub>(VOC)WI<sub></sub>AOSE<sub></sub>mAP<sub>K↑</sub>AP<sub>U↑</sub>Download
FR-CNNR-5080.0619.501651858.360config model
PROSERR-5079.4220.441426656.7216.99config model
ORER-5079.8018.181281158.252.60config model
DSR-5079.7016.761306258.468.75config model
OpenDetR-5080.0212.501075858.6414.38config model
OpenDetSwin-T83.2910.76914963.4216.35config model
MethodmAP<sub>K↑</sub>(VOC)WI<sub></sub>AOSE<sub></sub>mAP<sub>K↑</sub>AP<sub>U↑</sub>Download
RetinaNet79.6314.163653157.320config model
Open-RetinaNet79.6410.741720857.3210.55config model

Note:

Online Demo

Try our online demo at huggingface space.

Train and Test

First, you need to download pretrained weights in the model zoo, e.g., OpenDet.

Then, run the following command:

python tools/train_net.py --num-gpus 8 --config-file configs/faster_rcnn_R_50_FPN_3x_opendet.yaml \
        --eval-only MODEL.WEIGHTS output/faster_rcnn_R_50_FPN_3x_opendet/model_final.pth

The training process is the same as detectron2.

python tools/train_net.py --num-gpus 8 --config-file configs/faster_rcnn_R_50_FPN_3x_opendet.yaml

To train with the Swin-T backbone, please download swin_tiny_patch4_window7_224.pth and convert it to detectron2's format using tools/convert_swin_to_d2.py.

wget https://github.com/SwinTransformer/storage/releases/download/v1.0.0/swin_tiny_patch4_window7_224.pth
python tools/convert_swin_to_d2.py swin_tiny_patch4_window7_224.pth swin_tiny_patch4_window7_224_d2.pth

Citation

If you find our work useful for your research, please consider citing:

@InProceedings{han2022opendet,
    title     = {Expanding Low-Density Latent Regions for Open-Set Object Detection},
    author    = {Han, Jiaming and Ren, Yuqiang and Ding, Jian and Pan, Xingjia and Yan, Ke and Xia, Gui-Song},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    year      = {2022}
}