Home

Awesome

EP-SAM: Weakly Supervised Histopathology Segmentation via Enhanced Prompt with Segment Anything

Joonhyeon Song, Seohwan Yun, Seongho Yoon, Joohyeok Kim, Sangmin Lee by Kwangwoon University and QI LAB.

<a href='https://arxiv.org/abs/2410.13621'><img src='https://img.shields.io/badge/ArXiv-2410.13621-red' /></a>

Abstract

This work proposes a novel approach beyond supervised learning for effective pathological image analysis, addressing the challenge of limited robust labeled data. Pathological diagnosis of diseases like cancer has conventionally relied on the evaluation of morphological features by physicians and pathologists. However, recent advancements in compute-aided diagnosis (CAD) systems are gaining significant attention as diagnostic support tools. Although the advancement of deep learning has improved CAD significantly, segmentation models typically require large pixel-level annotated dataset, and such labeling is expensive. Existing studies not based on supervised approaches still struggle with limited generalization, and no practical approach has emerged yet. To address this issue, we present a weakly supervised semantic segmentation (WSSS) model by combining class activation map and Segment Anything Model (SAM)-based pseudo-labeling. For effective pretraining, we adopt the SAM—a foundation model that is pretrained on large datasets and operates in zero-shot configurations using only coarse prompts. The proposed approach utilizes explicit visual prompting to transfer enhanced Attention Dropout Layer's knowledge to SAM, thereby generating pseudo-labels. To demonstrate the superiority of the proposed method, experimental studies are conducted on histopathological breast cancer datasets. The proposed method outperformed other State-of-the-Art (SOTA) WSSS methods across three datasets, demonstrating its efficiency by achieving this with only 12GB of GPU memory during training.

Overview

The proposed approach transfer enhanced Attention Dropout Layer's knowledge to Segment Anything, thereby generating pseudo-labels.

Environment

The following describes our experimental setup.

Execute the following cell to set up a virtual environment for running the code.

# create a virtual env and activate
conda create -n EP_SAM python=3.10
conda activate EP_SAM 

# download packages
pip install -r requirements.txt

Additional package is required to extract patches from WSI. Install the following package to run the code:

How to Use

Train


cd code/camelyon16 # Move dataset directory and download Camelyon16bdatasets
cd ../camelyon17 # Move dataset directory and download Camelyon16bdatasets
cd ../train.sh

Inference


cd code/infer.sh

Dataset

We utilized open public breast cancer WSI datasets: Camelyon16 & Camelyon17.

Find detailed dataset descriptions in the links below. For download instructions and other details, see here.

Camelyon16

Camelyon17