Home

Awesome

This is the project page for paper:

You Only Segment Once: Towards Real-Time Panoptic Segmentation, In CVPR 2023.

<img width="90%" src="./demo/fig.png"/>

Model Zoo

On COCO validation set:

BackboneScalePQFPSGPUModel
R50800,133348.423.6V100model
R50512,80046.445.6V100model

On Cityscapes validation set:

BackboneScalePQFPSGPUModel
R501024,204859.711.1V100model
R50512,102452.522.6V100model

On ADE20k validation set:

BackboneScalePQFPSGPUModel
R50640,256038.035.4V100model

On Mapillary Vistas validation set:

BackboneScalePQFPSGPUModel
R502048,204834.17.1A100model

Getting Started

Installation

We recommend to use Anaconda for installation.

conda create -n YOSO python=3.8 -y
conda activate YOSO
conda install pytorch==1.10.1 torchvision==0.11.2 cudatoolkit=11.3 -c pytorch
pip install pycocotools -i https://pypi.douban.com/simple
pip install git+https://github.com/cocodataset/panopticapi.git
git clone https://github.com/hujiecpp/YOSO.git
cd YOSO
python setup.py develop

Datasets Preparation

See Preparing Datasets for Mask2Former.

Training & Evaluation

python projects/YOSO/train_net.py --num-gpus 4 --config-file projects/YOSO/configs/coco/panoptic-segmentation/YOSO-R50.yaml
python projects/YOSO/train_net.py --num-gpus 4 --config-file projects/YOSO/configs/coco/panoptic-segmentation/YOSO-R50.yaml --eval-only MODEL.WEIGHTS ./model_zoo/yoso_res50_coco.pth

Inference on Custom Image or Video

python demo/demo.py --config-file projects/YOSO/configs/coco/panoptic-segmentation/YOSO-R50.yaml --video-input input_video.mp4 --output output_video.mp4 --opts MODEL.WEIGHTS ./model_zoo/yoso_res50_coco.pth

Acknowledgements

Citing YOSO

If YOSO helps your research, please cite it in your publications:

@inproceedings{hu2023you,
  title={You Only Segment Once: Towards Real-Time Panoptic Segmentation},
  author={Hu, Jie and Huang, Linyan and Ren, Tianhe and Zhang, Shengchuan and Ji, Rongrong and Cao, Liujuan},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={17819--17829},
  year={2023}
}