Home

Awesome

MGCDA: Map-Guided Curriculum Domain Adaptation

Created by Christos Sakaridis at Computer Vision Lab, ETH Zurich.

MGCDA overview <br/><br/>

Overview

This is the source code for the MGCDA method for semantic segmentation at nighttime.

MGCDA: Paper | Dark Zurich Dataset | Challenge | Project | Conference Paper

MGCDA is presented in our IEEE TPAMI 2020 paper Map-Guided Curriculum Domain Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation and its original version GCMA was introduced in our ICCV 2019 paper.

For the source code for the uncertainty-aware semantic segmentation evaluation with the UIoU metric, you can consult the UIoU Dark Zurich Challenge page.

License

This software is made available for non-commercial use under a creative commons license. You can find a summary of the license here.

Contents

  1. Requirements
  2. Demo
  3. Testing
  4. Training
  5. Acknowledgments
  6. Citation

Requirements

For running the demo, you only need MATLAB 2016b or later.

For testing, you need:

  1. Linux
  2. NVIDIA GPU with CUDA & CuDNN
  3. MATLAB: version 2016b

For training, you need:

  1. Linux
  2. NVIDIA GPU with CUDA & CuDNN
  3. MATLAB: version 2016b
  4. Python 3

Demo

Run the demo MATLAB script.

This applies the geometrically guided segmentation refinement involved in MGCDA on a pair of corresponding images, i.e. a dark image and a daytime image which depict the same scene from a different viewpoint.

The results of the guided refinement, i.e. the refined segmentation of the dark image and the daytime segmentation aligned to the viewpoint of the dark image, are written in the directory output/demo/.

Testing MGCDA

You can also test MGCDA on other sets, such as Nighttime Driving, BDD100K-night (a selected nighttime subset of the segmentation set of BDD100K), and the validation set of Dark Zurich, simply by:

  1. downloading the respective set, similarly to above
  2. changing line 9 of the inner test script source/Semantic_segmentation/Experiments/Union_Cityscapes_Dark_Zurich/scripts/DarkCityscapes_DarkZurichNight_CycleGANfc-DarkZurich_twilight_labels_refinenet_init_geoRefDynDay-w_1-test_DarkZurich_testAnon.sh to the name of the respective set, e.g. to Nighttime_Driving. Consult the MATLAB testing function for a list of supported test sets.

To test MGCDA on your own custom set, you need to:

  1. implement a MATLAB function for your set similar to the function source/Semantic_segmentation/refinenet/main/my_gen_ds_info_Dark_Zurich_test_anon.m that corresponds to Dark Zurich-test
  2. augment the MATLAB testing function with a handle to the above function.

Training MGCDA

Acknowledgments

Our implementation includes adapted versions of two external repositories:

Citation

If you use our code in your work, please cite our publications as

@article{SDV20,
  author = {Sakaridis, Christos and Dai, Dengxin and Van Gool, Luc},
  title = {Map-Guided Curriculum Domain Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation}, 
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence}, 
  year = {2020},
  doi = {10.1109/TPAMI.2020.3045882}
}

and

@inproceedings{SDV19,
  author = {Sakaridis, Christos and Dai, Dengxin and Van Gool, Luc},
  title = {Guided Curriculum Model Adaptation and Uncertainty-Aware Evaluation for Semantic Nighttime Image Segmentation},
  booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
  year = {2019}
}

Contact

Christos Sakaridis
csakarid[at]vision.ee.ethz.ch
https://www.trace.ethz.ch/publications/2019/GCMA_UIoU