Awesome

AUCSeg

This repository is the official code for the paper " AUCSeg: AUC-oriented Pixel-level Long-tail Semantic Segmentation" (NeurIPS 2024).

Paper Title: AUCSeg: AUC-oriented Pixel-level Long-tail Semantic Segmentation.

Author: Boyu Han, Qianqian Xu*, Zhiyong Yang, Shilong Bao, Peisong Wen, Yangbangyan Jiang, Qingming Huang*

Installation

• Clone the repository

git clone https://github.com/boyuh/AUCSeg.git

• Install the required libraries according to the guidelines in MMSegmentation - get_started.
• Download ADE20K, cityscapes, and coco_stuff164k according to the guidelines in MMSegmentation - dataset_prepare.

• (Optional) If you want to use SegNeXt as the backbone, download the mscan-l pretrained model from Our - Google Drive or SegNeXt - TsingHua Cloud.

Training

AUCSeg is a plug-and-play component that can be integrated into almost any existing backbone with just two simple modifications:

1. Add the AUCLoss module (options include SquareAUCLoss, HingeAUCLoss, and ExpAUCLoss) to the loss_decode in the model.
2. Add the TMemoryBank module to the train_pipeline.

Below, we use SegNeXt-large as an example backbone to demonstrate how to incorporate AUCSeg for training:

For the ADE20K dataset, run this command:

python tools/train.py aucseg_configs/aucseg.segnext.large.512x512.ade.160k.py --work-dir results/aucseg.segnext.large.512x512.ade.160k --seed 0 --gpu-id 0

For the Cityscapes dataset, run this command:

python tools/train.py aucseg_configs/aucseg.segnext.large.1024x1024.city.160k.py --work-dir results/aucseg.segnext.large.1024x1024.city.160k --seed 0 --gpu-id 0

For the COCO-Stuff 164K dataset, run this command:

python tools/train.py aucseg_configs/aucseg.segnext.large.512x512.coco_stuff164k.80k.py --work-dir results/aucseg.segnext.large.512x512.coco_stuff164k.80k --seed 0 --gpu-id 0

We provide the Config and Checkpoint for these three datasets.

Evaluation

To evaluate the model, run this command:

./tools/dist_test.sh /path/to/config /path/to/checkpoint_file 1 --eval mIoU

Citation

If you find our work inspiring or use our codebase in your research, please cite our work.

@inproceedings{han2024aucseg,
    title={AUCSeg: AUC-oriented Pixel-level Long-tail Semantic Segmentation}, 
    author={Boyu Han and Qianqian Xu and Zhiyong Yang and Shilong Bao and Peisong Wen and Yangbangyan Jiang and Qingming Huang},
    booktitle={Advances in Neural Information Processing Systems},
    year={2024}
}

Contact

If you find any issues or plan to contribute back bug-fixes, please contact us by Boyu Han (Email: hanboyu23z@ict.ac.cn).

Acknowledgement

Our codes are based on MMSegmentaion v0.24.1, SegNeXt and XCurve v1.1.0.