Home

Awesome

UNIVERSAL ORGANIZER OF SEGMENT ANYTHING MODEL FOR UNSUPERVISED SEMANTIC SEGMENTATION

Abstract

Unsupervised semantic segmentation (USS) aims to achieve high-quality segmentation without manual pixel-level annotations. Existing USS models provide coarse category classification for regions, but the results often have blurry and imprecise edges. Recently, a robust framework called the segment anything model (SAM) has been proven to deliver precise boundary object masks. Therefore, this paper proposes a universal organizer based on SAM, termed as UO-SAM, to enhance the mask quality of USS models. Specifically, using only the original image and the masks generated by the USS model, we extract visual features to obtain positional prompts for target objects. Then, we activate a lolal region optimizer that performs segmentation using SAM on a per-object basis. Finally, we employ a global region optimizer to incorporate global image information and refine the masks to obtain the final fine-grained masks. Compared to existing methods, our UO-SAM achieves state-of-the-art performance.

img

性能分析

img img

How to run

  1. Creating a virtual environment in the terminal: conda create -n uosam python=3.8
  2. Installing necessary packages: pip install -r requirements.txt
  3. To speed up reasoning, install Mobile Segment Anything: 
    pip install git+https://github.com/ChaoningZhang/MobileSAM.git
    or clone the repository locally and install with 
    git clone git@github.com:ChaoningZhang/MobileSAM.git
cd MobileSAM; pip install -e .
  4. First download a model checkpoint. 
    python uo-sam.py  --data the/path/to/data --mask the/path/to/mask --outdir the/path/to/result
    For MobileSAM with higher efficiency, just add --sam_type vit_t: 
    python uo-sam.py  --data the/path/to/data --mask the/path/to/mask --outdir the/path/to/result --sam_type vit_t

致谢

此项目基于论文 Large-scale Unsupervised Semantic Segmentation,Causal unsupervised semantic segmentation 实现，部分代码参考了 SAM PASS CAUSE SmooSeg.