Home

Awesome

<div align="center"> <h1> Improving the Generalization of Segmentation Foundation Model under Distribution Shift via Weakly Supervised Adaptation </h1>

<a href='https://zhang-haojie.github.io/project-pages/wesam.html'><img src='https://img.shields.io/badge/Project-Page-green'></a> <a href='http://arxiv.org/abs/2312.03502'><img src='https://img.shields.io/badge/Technique-Report-red'></a>

</div>

🎈 News

🚀 Introduction

<div align="center"> <img width="800" alt="image" src="asserts/teaser.webp?raw=true"> </div>

Segment Anything Model was pre-trained on a large-scale dataset but exhibits awkward performance on diverse downstream segmentation tasks. We adapt SAM through weak supervision to enhance its generalization capabilities.

📻 Overview

<div align="center"> <img width="800" alt="image" src="asserts/Pipeline.webp?raw=true"> </div>

The proposed self-training architecture with anchor network regularization and contrastive loss regularization. Red arrows indicates the backpropagation flow.

📆 TODO

🎮 Getting Started

1. Install Environment

see INSTALL.

2. Prepare Dataset and Checkpoints

see PREPARE.

3. Adapt with Weak Supervision

# 1 modify configs/config.py 
# Prompt type: box, point, coarse

# 2 adapt
python adaptation.py

4. Validation

python validate.py --ckpt /path/to/checkpoint

🖼️ Visualization

<div align="center"> <img width="800" alt="image" src="asserts/VISUAL.webp?raw=true"> </div>

🎫 License

The content of this project itself is licensed under LICENSE.

💡 Acknowledgement

🖊️ Citation

If you find this project useful in your research, please consider cite:

@inproceedings{zhang2024improving,
  title={Improving the generalization of segmentation foundation model under distribution shift via weakly supervised adaptation},
  author={Zhang, Haojie and Su, Yongyi and Xu, Xun and Jia, Kui},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={23385--23395},
  year={2024}
}