Home

Awesome

Knowledge Transfer with Simulated Inter-Image Erasing for Weakly Supervised Semantic Segmentation

Network Architecture

The architecture of our proposed approach is as follows network

Prerequisite

conda env create -f wsss_new.yaml

Test KTSE

python infer.py --name ktse1 --model ktse --load_epo 39 --dict  --infer_list voc12/train_aug.txt
python evaluation.py --name ktse1 --task cam --dict_dir dict

Train KTSE

python train.py --name ktse1 --model ktse

Second stage training or testing for the segmentation network BECO

Testing the segmentation results with our pretrained model directly

Prerequisite for the segmentation task

cd segmentation
pip install -r requirements.txt 

python main.py --test --logging_tag seg_result --ckpt best_ckpt_KTSE_73.0.pth
python test.py --crf --logits_dir ./data/logging/seg_result/logits_msc --mode "val"

Refine the seed for pseudo label with IRN

cd irn 
python run_sample.py
python gen_mask.py

Training the segmentation network

Prepare the data directory

data/
    --- VOC2012/
        --- Annotations/
        --- ImageSet/
        --- JPEGImages/
        --- SegmentationClass/
        --- ...
    --- sem_seg/
        --- ****.png
        --- ****.png
    --- mask_irn/
        --- ****.png
        --- ****.png
    --- model_zoo/
        --- resnet-101_v2.pth
    --- logging/
cd segmentation
python main.py -dist --logging_tag seg_result --amp

Acknowledgements

This code is heavily borrowed from AEFT, IRN and BECO.

Citation

If you find this useful in your research, please consider citing:

@article{chen2024knowledge,
title={Knowledge Transfer with Simulated Inter-Image Erasing for Weakly Supervised Semantic Segmentation},
author={Chen, Tao and Jiang, Xiruo and Pei, Gensheng and Sun, Zeren and Wang, Yucheng and Yao, Yazhou},
journal={European Conference on Computer Vision (ECCV)},
year={2024}
}