Awesome
Evaluation Study on SAM 2 for Class-agnostic Instance-level Segmentation
Code repository for the paper titled "Evaluation Study on SAM 2 for Class-agnostic Instance-level Segmentation"
[Paper
]
Get Started
Install
- Download the SAM and SAM2 code from the website. [
SAM
] and [SAM2
] - Please follow the [
SAM
] and [SAM2
] to install the enveriment. - copy the code in SAM/notebooks into segment-anything/notebooks. Copy the code in SAM2/notebooks into segment-anythings/notebooks
Datasets
SIS:
- ILSO1K: Google Drive
- SOC: Google Drive
- SIS10KBaidu Disk Verification code: hust
- SIP: Google Drive
CIS:
- COD10K: Baidu (password:hust) / Google / Quark; Json files: Baidu (password:hust) / Google
- NC4K: Baidu (password:hust) / Google; Json files: Baidu (password:hust) / Google
SID:
- SOBA dataset can be downloaded from https://github.com/stevewongv/SSIS
DIS:
- DIS5K dataset can be downloaded from https://github.com/xuebinqin/DIS
Evaluation
SIS/CIS/SID:
- Download the weights of SAM and SAM2 from the website.
- Put weights into the checkpoints/
- Use .sh files in notebooks to evaluate:
-->use SAM, auto model, SIS task as example:\ modify the dataset root in SAM/notebooks/run_auto_saliency.sh
# SIS task / SAM / auto model
$ cd notebooks/
$ bash run_auto_saliency.sh
Note that, [sam/sam2]_[your mode]shadow.sh means the SID task. For CIS task, you can change the dataset path and save_json path in [sam/sam2][your mode]cos10k.py or [sam/sam2][your mode]_nc4k.py, then
python [sam/sam2]_[your mode]_cos10k.py
# or
python [sam/sam2]_[your mode]_nc4k.py
SIS task use he AP70 metric instead of AP75.
After running the [sam/sam2]_[your mode]_saliency.sh, please change the path in run_AP70.sh, then
$ bash run_AP70.sh
DIS:
- To Get the predicted results of SAM.
- automatic prompts mode:
following the [SAM
], masks can be generated for images from the command line:
python scripts/amg.py --checkpoint <path/to/checkpoint> --model-type <model_type> --input <image_or_folder> --output <path/to/output>
then, selecting the most suitable foreground mask, use a maximum Intersection over Union (IoU)
cd DIS/script
python3 findMaxIoUMask.py
- bounding box prompt mode:
cd DIS/SAM
python3 test_with_box_prompt_floder.py
- To Evaluate the predicted results.
cd DIS/metrics
python3 test_metrics.py
python3 hce_metric_main.py
Qualitative Results of SIS, CIS and SID
Citation
If you find our work useful for your research or applications, please cite using this BibTeX:
@article{zhang2024evalsam2,
title={Evaluation Study on SAM 2 for Class-agnostic Instance-level Segmentation},
author={Zhang, Tiantian and Zhou, Zhangjun and Pei, Jialun},
journal={arXiv preprint arXiv:2409.02567},
year={2024},
url={https://arxiv.org/abs/2409.02567}
}
Acknowledgement
Thanks for the efforts of the authors involved in the Segment Anything, Segment Anything 2 and UnderwaterSAM2Eval.