Awesome

<h2 align="center"> <a href="">RecLMIS: Cross-Modal Conditioned Reconstruction for Language-guided Medical Image Segmentation</a></h2> <a src="https://img.shields.io/badge/cs.CV-2312.09278-b31b1b?logo=arxiv&logoColor=red" href="https://arxiv.org/pdf/2404.02845"> <img src="https://img.shields.io/badge/cs.CV-2404.02845-b31b1b?logo=arxiv&logoColor=red"> </a> <h5 align="center"> 🍒🍒🍒 This paper was accepted by IEEE Transactions on Medical Imaging (TMI). If you like our project, please give us a star ⭐ on GitHub for latest update.

😮 Hightlights

✨ We present an efficient language-guided medical image segmentation framework, which employs conditioned contrastive learning and cross-modal conditioned reconstruction to achieve fine-grained cross-modal alignment.
✨ We introduce the conditioned interaction module. It focuses on the key feature token information aligned between images and text, and we have demonstrated that can reduce interference from background information present in both images and text.
✨ We propose the Conditioned Language Reconstruction module and Conditioned Vision Reconstruction. Both of them utilize cross-modal masking features for reconstruction, and we have demonstrated that could enhance the representational capacity of the visual encoder.
✨ Extensive experiments show that the proposed RecLMIS outperforms the state-of-the-art methods on widely used public benchmarks, exhibiting more accurate results and faster inference speed.

🔥 Updates

2024-12-23: We release the code.
2024-04-05: We release the paper and code link.

Installation
Dataset
Train
Test
Acknowledgement
License
Citation

🛠️Installation

Clone this repository and navigate to RecLMIS folder

git clone https://github.com/ShawnHuang497/RecLMIS.git
cd RecLMIS

Install Package

conda create -n reclmis python=3.9 -y
conda activate reclmis
pip install --upgrade pip 
pip install -r requirements.txt

Download pretrained CLIP: ViT-B-32.pt, and put it in the folder nets/

🗃️Dataset

You can refer to MedPLIB to download the dataset.
If your dataset is not in current path or disk, you can modify the path in Config_xxx.py file or ues ln -s {old_path} {./datasets} to create a soft link to link the data to current path.

📀Train

sh train.sh 0 Config_xxx

🥭 Test

python test.py --cfg_path Config_xxx --test_session session_09.25_00h27 --gpu {0} --test_vis {True}

👍Acknowledgement

This code is based on LVIT, ViT and CLIP.

🔒License

The majority of this project is released under the Apache 2.0 license as found in the LICENSE file.

✏️Citation

If you find our paper and code useful in your research, please consider giving a star and citation.

@article{huang2024cross,
  title={Cross-Modal Conditioned Reconstruction for Language-guided Medical Image Segmentation},
  author={Huang, Xiaoshuang and Li, Hongxiang and Cao, Meng and Chen, Long and You, Chenyu and An, Dong},
  journal={arXiv preprint arXiv:2404.02845},
  year={2024}
}