Awesome

Awesome-Segment-Anything

Collect some resource about Segment Anything (SAM), including latest papers and demo.

Papers

Segment Anything original paper

• Segment Anything [arXiv 2023]

Technical blog

• [English Blog] Introducing Segment Anything: Working toward the first foundation model for image segmentation [Link]
• [Chinese Blog] 论文解读MetaAi SAM分割一切 [Link]

Discussion

• [Reddit] Meta AI has released both the Model AND the dataset for Segment Anything [Link]
• [Zhihu] Meta 发布图像分割论文 Segment Anything，将给 CV 研究带来什么影响？[Link]

arXiv papers

• SAM Fails to Segment Anything? -- SAM-Adapter: Adapting SAM in Underperformed Scenes: Camouflage, Shadow, and More [paper] [code]
• Learning to "Segment Anything" in Thermal Infrared Images through Knowledge Distillation with a Large Scale Dataset SATIR [paper] [code]
• The Segment Anything foundation model achieves favorable brain tumor autosegmentation accuracy on MRI to support radiotherapy treatment planning [paper]
• Deep learning universal crater detection using Segment Anything Model (SAM) [paper]
• Can SAM Segment Polyps? [paper] [code]
• Inpaint Anything: Segment Anything Meets Image Inpainting [paper] [code]
• [SEEM] Segment Everything Everywhere All at Once [paper] [code]
• SAM Struggles in Concealed Scenes -- Empirical Study on "Segment Anything" [paper]
• Segment Anything Is Not Always Perfect: An Investigation of SAM on Different Real-world Applications [paper]
• CLIP Surgery for Better Explainability with Enhancement in Open-Vocabulary Tasks [paper]
• SAMM (Segment Any Medical Model): A 3D Slicer Integration to SAM [paper]
• SAM.MD: Zero-shot medical image segmentation capabilities of the Segment Anything Model [paper]
• Brain Extraction comparing Segment Anything Model (SAM) and FSL Brain Extraction Tool [paper]
• Can SAM Segment Anything? When SAM Meets Camouflaged Object Detection [paper]
• Segment Anything Model (SAM) for Digital Pathology: Assess Zero-shot Segmentation on Whole Slide Imaging [paper]

Application

Image Detection/Segmentation

• [Grounded-Segment-Anything]: A very interesting demo by combining Grounding DINO and Segment Anything

• [GroundedSAM-zero-shot-anomaly-detection]: Segment any anomaly without any training

• [Semantic Segment Anything]: SSA is an automated annotation engine that serves as the initial semantic labeling for the SA-1B dataset.

• [Segment Anything with Clip]: It aims to resolve downstream segmentation tasks with prompt engineering, such as foreground/background points, bounding box, mask, and free-formed text.

• [Prompt-Segment-Anything]: This is an implementation of zero-shot instance segmentation using Segment Anything.

• [SAM-RBox]: This is an implementation of SAM (Segment Anything Model) for generating rotated bounding boxes with MMRotate.

• [Open-vocabulary-Segment-Anything]: An interesting demo by combining OWL-ViT of Google and Segment Anything of Meta!

• [SegDrawer]: Simple static web-based mask drawer, supporting semantic drawing with Segment Anything Model.

• [MetaSeg: Packaged version of the Segment Anything repository]: This repo is a packaged version of the segment-anything model.

• [Segment Anything EO tools]: This tools are developed to ease the processing of spatial data (GeoTIFF and TMS) with Meta AI Segment Anything models using sliding window algorithm for big files.

• [SEEM]: SEEM allows users to easily segment an image using prompts of different types including visual prompts (points, marks, boxes, scribbles and image segments) and language prompts (text and audio), etc

Distillation and Automated Labeling

• Autodistill: Images to inference with no labeling (use foundation models to train supervised models). autodistill features a autodistill-grounded-sam module that enables automated image annotation using Grounding DINO and SAM.

3D

• [3D-Box via Segment Anything]: In this project, we extend the scope to 3D world by combining Segment Anything and VoxelNeXt. When we provide a prompt (e.g., a point / box), the result is not only 2D segmentation mask, but also 3D boxes.

• [Anything-3DNovel-View]: Combining Segment Anything and a series of 3D models

Labeling

• [AnyLabeling]: AnyLabeling = LabelImg + Labelme + Improved UI + Auto-labeling

Image Generation

• [Segment Anything for Stable Diffusion WebUI]: This extension aim for helping stable diffusion webui users to use segment anything to do stable diffusion inpainting.

• [IEA: Image Editing Anything]: Using stable diffusion and segmentation anything models for image editing.

• [Edit Anything by Segment-Anything]: This is an ongoing project aims to Edit and Generate Anything in an image, powered by Segment Anything, ControlNet, BLIP2, Stable Diffusion, etc.

• [Inpaint Anything]: Segment Anything Meets Image Inpainting

• [Magic Copy]: Magic Copy is a Chrome extension that uses Meta's Segment Anything Model to extract a foreground object from an image and copy it to the clipboard.