Awesome

Weakly-supervised Instance Segmentation via Class-agnostic Learning with Salient Images.(CVPR 2021)

This is the official code of the paper Weakly-supervised Instance Segmentation via Class-agnostic Learning with Salient Images, published in CVPR 2021, by Xinggang Wang, Jiapei Feng, Bin Hu, Qi Ding, Longjin Ran, Xiaoxin Chen, Wenyu Liu.

Introduction

Weakly-supervised instance segmentation uses coarser annotations to acquire a high-precision instance segmentation model, such as bounding boxes. This "box-to-seg" process is a class-agnostic process. Our motivation is to learn a model that achieves generic class-agnostic segmentation.

The training process is divided into three steps. Firstly, we obtain a class-agnostic segmentation model through the joint training of box-supervised dataset and salient object segmentation dataset. Secondly, we use the class-agnostic segmentation model to produce high-quality predictions for training instances. Those segmentation predictions are merged to generate proxy instance masks for training images. Finally, we re-train a Mask R-CNN with proxy masks.

Prerequisites

This is an example of how to list things you need to use the software and how to install them.

• Python>=3.6, PyTorch
• Augmented PASCAL VOC 2012 Dataset (10582 images for training)
• DUT-TR-Single Dataset

Usage

Here, we provide the DUT-TR-Single datasets, the cocostyle annotations of PASCAL VOC, our pre-trained models and proxy masks for training set.

• BaiduCloud The access code is 'yint'.
• GoogleDrive

Training

• After downloading the dataset, put them in the specific folder. Then, cd jointraining and run the following command to do joint training:

    bash train_pd.sh

for evaluation:

    bash pascal_val.sh

for predicting segmentation maps of the training instances:

    bash pascal_psd_mask.sh

More detailed instructions are provided in the jointraining.

• Generate the proxy masks for box-supervised dataset. Run the following command:

    cd proxy_mask
    python pascal_proxy_mask_generate.py --gt-path training_set_boundingbox_cocostyle_json --seg-pred predicted_results

• Retrain a Mask R-CNN. We use the Mask R-CNN as our instance segmentation framework and we modify two file include ../maskrcnn_benchmark/data/datasets/coco.py and ../maskrcnn_benchmark/modeling/roi_heads/mask_head/loss.py.

Run the following command:

    cd retrain/maskrcnn-benchmark/
    python -m torch.distributed.launch --nproc_per_node=2 ./tools/train_net.py --config-file e2e_mask_rcnn_R_101_FPN_4x_voc_aug_cocostyle.yaml

Check INSTALL.md for installation instructions.

Citation

If you find the code useful in your research, please consider citing:

@inproceedings{wang2021boxcaseg,
  title     =  {Weakly-supervised Instance Segmentation via Class-agnostic Learning with Salient Images},
  author    =  {Wang, Xinggang and Feng, Jiapei and Hu, Bin and Ding, Qi and Ran, Longjin and Chen, Xiaoxin and Liu, Wenyu},
  booktitle =  {Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR)},
  year      =  {2021}
}

License

Distributed under the MIT License. See LICENSE for more information.

Acknowledgements

• We thank NSFC, Zhejiang Laboratory and VIVO Inc for their support to this project.
• The code is borrowed from HRNet, BBTP and Mask R-CNN.