Home

Awesome

Unpair-Seg: Open-Vocabulary Segmentation with Unpaired Mask-Text Supervision

This repo contains the code for our paper UnpairSeg. It is a weakly supervised open-vocabulary segmentation framework that leverages unpaired mask-text pairs.

<!-- Our code will be released soon! -->

Now, we release the inference code and checkpoints for stage one training.

Installation

Inference

The part provides a brief introduction of the usage of Unpair-Seg. Please download the checkpoint of stage one training. We provide ./demo/inference.py for point-promptable segmentation. Run it with:

cd demo/
python inference.py \
    -c ../configs/S1_point_seg.yaml \
    -i ../images/*.jpg \
    --opt MODEL.WEIGHTS stage1.pth

We also provide some test images under ./images/.


If you use this codebase, or otherwise found our work valuable, please cite:

@article{wang2024open,
  title={Open-Vocabulary Segmentation with Unpaired Mask-Text Supervision},
  author={Wang, Zhaoqing and Xia, Xiaobo and Chen, Ziye and He, Xiao and Guo, Yandong and Gong, Mingming and Liu, Tongliang},
  journal={arXiv preprint arXiv:2402.08960},
  year={2024}
}