Awesome
PSALM: Pixelwise SegmentAtion with Large Multi-Modal Model
Zheng Zhang*, Yeyao Ma*, Enming Zhang*, Xiang Bai
<sup>* Equal Contribution
Features
- A powerful extension of the Large Multi-modal Model for generic (panoptic, instance, semantic) segmentation, referring segmentation and interactivate segmentation.
- Support joint training across multiple segmentation tasks and visual-language tasks.
- Demonstrates zero-shot capabilities on unseen task, such as open-vocabulary segmentation, generalizaed referring segmentation, and video object segmentation.
Updates
- Release evaluation code
- Release training code
Installation
See Installation instructions.
Getting Started
See Preparing Datasets for PSALM.
See Getting Started with PSALM.
Model Zoo
- Download PSALM here.
Citation
If you think this work is useful for your research, please use the following BibTeX entry.
@misc{zhang2024psalm,
title={PSALM: Pixelwise SegmentAtion with Large Multi-Modal Model},
author={Zheng Zhang and Yeyao Ma and Enming Zhang and Xiang Bai},
year={2024},
eprint={2403.14598},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
Acknowledgement
Thanks for awesome works: Mask2former, Mask2former-Simplify and LLaVA. Code is based on these works.