Home

Awesome

RefineMask: Towards High-Quality Instance Segmentation <br>with Fine-Grained Features (CVPR 2021)

This repo is the official implementation of RefineMask: Towards High-Quality Instance Segmentation with Fine-Grained Features.

Update: A faster and slightly better implementation is available! You can train R50-RefineMask 2 images per gpu with less than 11G memory cost now. See here.

Framework

image

Main Results

Results on COCO

MethodBackboneScheduleAPAP<sup>*</sup>Checkpoint
Mask R-CNNR50-FPN1x34.736.8
RefineMaskR50-FPN1x37.340.6download
Mask R-CNNR50-FPN2x35.437.7
RefineMaskR50-FPN2x37.841.2download
Mask R-CNNR101-FPN1x36.138.4
RefineMaskR101-FPN1x38.641.8download
Mask R-CNNR101-FPN2x36.639.3
RefineMaskR101-FPN2x39.042.4download

Note: No data augmentations except standard horizontal flipping were used.

Results on LVIS

MethodBackboneScheduleAPAP<sub>r</sub>AP<sub>c</sub>AP<sub>f</sub>Checkpoint
Mask R-CNNR50-FPN1x22.110.121.730.0
RefineMaskR50-FPN1x25.713.824.931.8download
Mask R-CNNR101-FPN1x23.712.323.229.1
RefineMaskR101-FPN1x27.115.626.233.1download

Results on Cityscapes

MethodBackboneScheduleAPAP<sub>S</sub>AP<sub>M</sub>AP<sub>L</sub>Checkpoint
Mask R-CNNR50-FPN1x33.812.031.551.8
RefineMaskR50-FPN1x37.614.035.457.9download

Efficiency of RefineMask

MethodAPAP<sup>*</sup>FPS
Mask R-CNN34.736.815.7
PointRend35.638.711.4
HTC37.440.74.4
RefineMask37.340.911.4

Usage

Requirements

Datasets

data
  ├── coco
  |   ├── annotations
  │   │   │   ├── instances_train2017.json
  │   │   │   ├── instances_val2017.json
  │   │   │   ├── lvis_v0.5_val_cocofied.json
  │   ├── train2017
  │   │   ├── 000000004134.png
  │   │   ├── 000000031817.png
  │   │   ├── ......
  │   ├── val2017
  │   ├── test2017
  ├── lvis
  |   ├── annotations
  │   │   │   ├── lvis_v1_train.json
  │   │   │   ├── lvis_v1_val.json
  │   ├── train2017
  │   │   ├── 000000004134.png
  │   │   ├── 000000031817.png
  │   │   ├── ......
  │   ├── val2017
  │   ├── test2017
  ├── cityscapes
  |   ├── annotations
  │   │   │   ├── instancesonly_filtered_gtFine_train.json
  │   │   │   ├── instancesonly_filtered_gtFine_val.json
  │   ├── leftImg8bit
  │   |   ├── train
  │   │   ├── val
  │   │   ├── test

Note: We used the lvis-v1.0 dataset which consists of 1203 categories.

Training

./scripts/dist_train.sh ./configs/refinemask/coco/r50-refinemask-1x.py 8 work_dirs/r50-refinemask-1x

Note: <strike>The codes only support batch size 1 per GPU, and we trained all models with a total batch size 16x1. If you train models with a total batch size 8x1, the performance may drop. We will support batch size 2 or more per GPU later. You can use ./scripts/slurm_train.sh for training with multi-nodes.</strike> Multiple images per GPU during training has been supported now.

Inference

./scripts/dist_test.sh ./configs/refinemask/coco/r50-refinemask-1x.py 8 work_dirs/r50-refinemask-1x

Citation

@InProceedings{Zhang_2021_CVPR,
    author    = {Zhang, Gang and Lu, Xin and Tan, Jingru and Li, Jianmin and Zhang, Zhaoxiang and Li, Quanquan and Hu, Xiaolin},
    title     = {RefineMask: Towards High-Quality Instance Segmentation With Fine-Grained Features},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2021},
    pages     = {6861-6869}
}