Home

Awesome

Feature-prompting GBMSeg: One Shot Reference Guided Training-Free Feature Matching for Glomerular Basement Membrane Segmentation and Quantification

<br>Xueyu Liu, Guangze Shi, Rui Wang, Yexin Lai, Jianan Zhang, Lele Sun, Quan Yang, Yongfei Wu*, Weixia Han, Ming Li, and Wen Zheng<br> <sup>1</sup>Taiyuan University of Technology,   <sup>2</sup>The Second Affiliated Hospital of Shanxi Medical University,  <sup>3</sup>Shanxi Provincial People's Hospital

πŸš€πŸš€This work has been accepted by MICCAI2024!πŸš€πŸš€

We present GBMSeg, a training-free framework that automates the segmentation and measurement of the glomerular basement membrane (GBM) in TEM using only one-shot reference images. GBMSeg leverages the robust feature matching capabilities of pretrained foundation models (PFMs) to generate initial prompts, designs novel prompting engineering for optimized prompting methods, and utilizes a class-agnostic segmentation model to obtain the final segmentation result.

<p align="center"> <img width="800" alt="ablation" src="img/ablation.png"> </p>

Usage

Setup

Datasets

../                          # parent directory
β”œβ”€β”€ ./data                   # data path
β”‚   β”œβ”€β”€ reference_image      # the one-shot reference image
β”‚   β”œβ”€β”€ reference_mask       # the one-shot reference mask
β”‚   β”œβ”€β”€ target_image         # testing images

Generate prompt

cd GBMSeg/feature-matching
python generate_prompt.py

Automatic prompt engineering

cd GBMSeg/tools
python automatic_prompt_engineering.py

Segmentation

mkdir GBMSeg/results
cd GBMSeg/segmenting-anything
python segment.py

Citation

If you find this project useful in your research, please consider citing:

@article{liu2024feature,
  title={Feature-prompting GBMSeg: One-Shot Reference Guided Training-Free Prompt Engineering for Glomerular Basement Membrane Segmentation},
  author={Liu, Xueyu and Shi, Guangze and Wang, Rui and Lai, Yexin and Zhang, Jianan and Sun, Lele and Yang, Quan and Wu, Yongfei and Li, MIng and Han, Weixia and others},
  journal={arXiv preprint arXiv:2406.16271},
  year={2024}
}

Acknowledgement

Thanks DINOv2, SAM. for serving as building blocks of GBMSeg.