Awesome

3D-MIR: A Benchmark & Empirical Study on 3D Medical Image Retrieval in Radiology

Paper: https://arxiv.org/pdf/2311.13752.pdf

Data:

• Images: We use Medical Segmentation Decathlon corresponding to four organs: Colon, Liver, Lung, and Pancreas.
• 3D-MIR labels and generated captions: https://github.com/abachaa/3D-MIR/tree/main/Data/3DMIR_labels
• Training/test splits: https://github.com/abachaa/3D-MIR/tree/main/Data

Code:

1) Organ Segmentation: We use Total Segmentator to segment and index individual organs.

• (a) Organ Segmentation: run_totalsegmentator.py

2) Image Processing and Quantification

• (a) Liver Quantification: msd_colon_2D_3D_metrics_extraction.ipynb

• (b) Pancreas Quantification: msd_pancreas_2D_3D_metrics_extraction.ipynb

• (c) Colon Quantification: msd_colon_2D_3D_metrics_extraction.ipynb

• (d) Lung Quantification: msd_lung_2D_3D_metrics_extraction.ipynb

3) Embeddings Generation (BiomedCLIP)

• Generate_BiomedCLIP_Embeddings.ipynb

4) Retrieval Methods & Evaluation:

• (a) Slice-based Retrieval: Method-1-Slice-based-Retrieval.ipynb (described in Section 4.1)

• (b) Volume-based Retrieval: Method-2-Volume-based-Retrieval.ipynb (described in Section 4.2)

• (c) Multi-modal Retrieval: Method-3-Multimodal-Retrieval.ipynb (described in Section 4.3) and GPT-4-based captions