Awesome
SeCo
Semantic Connectivity-Driven Pseudo-labeling for Cross-domain Segmentation.
:notebook_with_decorative_cover: For more information, please see our paper at ARXIV
:speech_balloon: Pixel-driven PL. v.s. Connectivity-driven PL
:speech_balloon: Pipeline
:speech_balloon: Requirements
Python 3.8.0
pytorch 1.10.1
torchvision 0.11.2
einops 0.3.2
Please see requirements.txt
for all the other requirements.
You can use PSA and SCC to obtain high-purity connectivity-based pseudo-labels. These pseudo-labels can then be exploited and embedded into existing unsupervised domain adaptative semantic segmentation methods.
:speech_balloon: Pixel Semantic Aggregation
First, you can obtain pixel-level pseudo-labels by pixel thresholding (e.g.cbst ) from a UDA method (e.g. ProDA ) or a source-free UDA method (e.g. DTST), or a UDG method (e.g. SHADE ).
And organize them in the following format.
"""
├─image
├─pixel-level pseudo-label
└─list
"""
list (XXX.txt) records the image names (XXX.png) and their corresponding pixel-level pseudo-labels.
Then, run the PSA as follows:
${exp_name}="HRDA_seco"
CUDA_VISIBLE_DEVICES="1" nohup python seco_sam.py --id-list-path ./splits/cityscapes/${exp_name}/all.txt --class-num ${class_name} > logs/${exp_name} 2>&1 &
Afterward, you can find the aggregated pseudo-labels in root_path/${exp_name}_vit_{B/H}
.
:speech_balloon: Semantic Connectivity Correction
After PSA, the noise is also amplified, and then you can use SCC to denoise the connected regions. Refer to (SCC ) part for specific instructions.
:speech_balloon: Embedded SeCo into UDA semantic segmentation
In the paper, we leverage the pseudo-labels generated by SeCo across multiple codebases. Due to the diversity in code structures, we provide a unified implementation for utilizing SeCo's pseudo-labels. This involves partitioning the unlabeled data into two subsets (one as a labeled subset and the other as an unlabeled subset) and employing a semi-supervised method (Unimatch ) for further adaptation, as mentioned in the paper.
Adaptation task | model | Before adaptation | After adaptation | Training logs |
---|---|---|---|---|
GTA → Cityscapes | deeplab-r101 | 55.1 DTST | 64.6 | training_logs/seco_gta |
Synthia → Cityscapes | deeplab-r101 | 52.3 DTST | 59.2 | training_logs/seco_synthia |
GTA → BDD100K | deeplab-r101 | 37.9 SFOCDA | 44.3 | training_logs/seco_bdd |
This table shows the source-free setting, which is more competitive than the report results in Tables 1 and 2 in the paper. More adaptation scripts will be updated later.
:speech_balloon: License
Code is released for non-commercial and research purposes only. For commercial purposes, please contact the authors.
Acknowledgement
Many thanks to those wonderful work and the open-source code.
:speech_balloon: Citation
If you use this code for your research, please cite our paper:
@misc{zhao2023semantic,
title={Semantic Connectivity-Driven Pseudo-labeling for Cross-domain Segmentation},
author={Dong Zhao and Ruizhi Yang and Shuang Wang and Qi Zang and Yang Hu and Licheng Jiao and Nicu Sebe and Zhun Zhong},
year={2023},
eprint={2312.06331},
archivePrefix={arXiv},
primaryClass={cs.CV}
}