Home

Awesome

CrDoCo: Pixel-level Domain Transfer with Cross-Domain Consistency

This repository contains the code for the paper CrDoCo: Pixel-level Domain Transfer with Cross-Domain Consistency.

<img src="img/teaser.png" width="1000">

Abstract

Unsupervised domain adaptation algorithms aim to transfer the knowledge learned from one domain to another (e.g., synthetic to real images). The adapted representations often do not capture pixel-level domain shifts that are crucial for dense prediction tasks (e.g., semantic segmentation). In this paper, we present a novel pixel-wise adversarial domain adaptation algorithm. By leveraging image-to-image translation methods for data augmentation, our key insight is that while the translated images between domains may differ in styles, their predictions for the task should be consistent. We exploit this property and introduce a cross-domain consistency loss that enforces our adapted model to produce consistent predictions. Through extensive experimental results, we show that our method compares favorably against the state-of-the-art on a wide variety of unsupervised domain adaptation tasks.

Citation

If you find our code useful, please consider citing our work using the following bibtex:

@inproceedings{CrDoCo,
  title={CrDoCo: Pixel-level Domain Transfer with Cross-Domain Consistency},
  author={Chen, Yun-Chun and Lin, Yen-Yu and Yang, Ming-Hsuan and Huang, Jia-Bin },
  booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2019}
}

Environment

Dataset

Demo code

We prepare a demo code so that you can have a better understanding on the workflow of the code. Please refer to demo.py

Pre-training the image-to-image translation network

We use the source code from Zhu et al.. Please follow the training tips here for pre-training the image-to-image translation networks. Please adjust the image size based on the GPU memory.

Note:

We found that the training of image-to-image translation network is not very stable since the domain gap between synthetic datasets and real-world datasets is large. Sometimes, the results will have severe visual artifacts or distortions. Based on our experience, we found that resizing images to resolution higher than 384x384 will implicitly alleviate this issue.

Training the task network

python train.py --model train --img_source_file /path/to/source/dataset --img_target_file /path/to/target/dataset --lab_source_file /path/to/source/label --lab_target_file /path/to/target/label --shuffle --flip --rotation

Evaluation

python test.py --model test --img_source_file /path/to/source/dataset --img_target_file /path/to/target/dataset

Acknowledgement