Awesome
Conect Segment-Anything with CLIP
We aim to classify the output masks of segment-anything with the off-the-shelf CLIP models. The cropped image corresponding to each mask is sent to the CLIP model. <img src="https://github.com/PengtaoJiang/SAM-CLIP/blob/main/imgs/pipeline.png" width="100%" height="50%">
Other Nice Works
Editing-Related Works
- sail-sg/EditAnything
- IDEA-Research/Grounded-Segment-Anything
- geekyutao/Inpaint-Anything
- Luodian/RelateAnything
Nerf-Related Works
- ashawkey/Segment-Anything-NeRF
- Anything-of-anything/Anything-3D
- Jun-CEN/SegmentAnyRGBD
- Pointcept/SegmentAnything3D
Segmentation-Related Works
- maxi-w/CLIP-SAM
- Curt-Park/segment-anything-with-clip
- kadirnar/segment-anything-video
- fudan-zvg/Semantic-Segment-Anything
- continue-revolution/sd-webui-segment-anything
- RockeyCoss/Prompt-Segment-Anything
- ttengwang/Caption-Anything
- ngthanhtin/owlvit_segment_anything
- lang-segment-anything
- helblazer811/RefSAM
- Hedlen/awesome-segment-anything
- ziqi-jin/finetune-anythin
- ylqi/Count-Anything
- xmed-lab/CLIP_Surgery
- RockeyCoss/Prompt-Segment-Anything
- segments-ai/panoptic-segment-anything
- Cheems-Seminar/grounded-segment-any-parts
- aim-uofa/Matcher
- SysCV/sam-hq
- CASIA-IVA-Lab/FastSAM
- ChaoningZhang/MobileSAM
- JamesQFreeman/Sam_LoRA
- UX-Decoder/Semantic-SAM
- cskyl/SAM_WSSS
- ggsDing/SAM-CD
- yformer/EfficientSAM
- XiaRho/SEMat
Labelling-Related Works
Tracking-Related Works
Medical-Related Works
Todo
- We plan to connect segment-anything with MaskCLIP.
- We plan to finetune on the COCO and LVIS datasets.
Run Demo
Download the sam_vit_h_4b8939.pth model from the SAM repository and put it at ./SAM-CLIP/
. Follow the instructions to install segment-anything and clip packages using the following command.
cd SAM-CLIP; pip install -e .
pip install git+https://github.com/openai/CLIP.git
Then run the following script:
sh run.sh
Example
Input an example image and a point (250, 250) to the SAM model. The input image and output three masks as follows:
<center><img src="https://github.com/PengtaoJiang/SAM-CLIP/blob/main/imgs/ADE_val_00000001.jpg" width="50%" height="50%"></center>The three masks and corresponding predicted category are as follows:
<center> <img src="https://github.com/PengtaoJiang/SAM-CLIP/blob/main/outs/ADE_val_00000001/outs.png" width="100%" height="50%"> </center> <center> <img src="https://github.com/PengtaoJiang/SAM-CLIP/blob/main/outs/ADE_val_00000001/logits.png" width="100%" height="50%"> </center>You can change the point location at L273-274 of scripts/amp_points.py
.
## input points
input_points_list = [[250, 250]]
label_list = [1]