Home

Awesome

Evaluation Study on SAM 2 for Class-agnostic Instance-level Segmentation

Code repository for the paper titled "Evaluation Study on SAM 2 for Class-agnostic Instance-level Segmentation"

[Paper]

SIS/CIS/SID

DIS

Get Started

Install

  1. Download the SAM and SAM2 code from the website. [SAM] and [SAM2]
  2. Please follow the [SAM] and [SAM2] to install the enveriment.
  3. copy the code in SAM/notebooks into segment-anything/notebooks. Copy the code in SAM2/notebooks into segment-anythings/notebooks

Datasets

SIS

CIS

SID

DIS

Evaluation

SIS/CIS/SID:

  1. Download the weights of SAM and SAM2 from the website.
  2. Put weights into the checkpoints/
  3. Use .sh files in notebooks to evaluate:

-->use SAM, auto model, SIS task as example:\ modify the dataset root in SAM/notebooks/run_auto_saliency.sh

# SIS task / SAM / auto model
$ cd notebooks/
$ bash run_auto_saliency.sh

Note that, [sam/sam2]_[your mode]shadow.sh means the SID task. For CIS task, you can change the dataset path and save_json path in [sam/sam2][your mode]cos10k.py or [sam/sam2][your mode]_nc4k.py, then

python  [sam/sam2]_[your mode]_cos10k.py
# or 
python  [sam/sam2]_[your mode]_nc4k.py

SIS task use he AP70 metric instead of AP75.

After running the [sam/sam2]_[your mode]_saliency.sh, please change the path in run_AP70.sh, then

$ bash run_AP70.sh

DIS:

  1. To Get the predicted results of SAM.

following the [SAM], masks can be generated for images from the command line:

python scripts/amg.py --checkpoint <path/to/checkpoint> --model-type <model_type> --input <image_or_folder> --output <path/to/output>

then, selecting the most suitable foreground mask, use a maximum Intersection over Union (IoU)

cd DIS/script
python3 findMaxIoUMask.py 
cd DIS/SAM
python3 test_with_box_prompt_floder.py 
  1. To Evaluate the predicted results.
cd DIS/metrics
python3 test_metrics.py 
python3 hce_metric_main.py

Qualitative Results of SIS, CIS and SID

  1. SAM2 sam2_demo
  2. SAM sam_demo

Citation

If you find our work useful for your research or applications, please cite using this BibTeX:

@article{zhang2024evalsam2,
      title={Evaluation Study on SAM 2 for Class-agnostic Instance-level Segmentation}, 
      author={Zhang, Tiantian and Zhou, Zhangjun and Pei, Jialun},
      journal={arXiv preprint arXiv:2409.02567},
      year={2024},
      url={https://arxiv.org/abs/2409.02567} 
}

Acknowledgement

Thanks for the efforts of the authors involved in the Segment Anything, Segment Anything 2 and UnderwaterSAM2Eval.