Awesome

Delving Deep into Intrinsic Image Harmonization

Here we provide the PyTorch implementation and pre-trained model of our latest version, if you require the code of our previous CVPR version ("Intrinsic Image Harmonization"), please click the release version.

Prerequisites

• Linux
• Python 3
• CPU or NVIDIA GPU + CUDA CuDNN

Datasets

• Download iHarmony4 dataset, and our HVIDIT dataset Google Drive or BaiduCloud (access code: akbi).

Base Model

• Train

CUDA_VISIBLE_DEVICES=0 python train.py --model iih_base  --name iih_base_allihd_test  --dataset_root <dataset_dir> --dataset_name IHD --batch_size xx --init_port xxxx

• Test

CUDA_VISIBLE_DEVICES=0 python test.py --model iih_base  --name iih_base_allihd_test  --dataset_root <dataset_dir> --dataset_name IHD --batch_size xx --init_port xxxx

• Apply pre-trained model

Download pre-trained model from Google Drive or BaiduCloud (access code: n4js), and put latest_net_G.pth in the directory checkpoints/iih_base_allihd. Run:

CUDA_VISIBLE_DEVICES=0 python test.py --model iih_base  --name iih_base_allihd  --dataset_root <dataset_dir> --dataset_name IHD --batch_size xx --init_port xxxx

Base Model with Lighting

• Train

CUDA_VISIBLE_DEVICES=0 python train.py --model iih_base_lt  --name iih_base_lt_allihd_test  --dataset_root <dataset_dir> --dataset_name IHD --batch_size xx --init_port xxxx

• Test

CUDA_VISIBLE_DEVICES=0 python test.py --model iih_base_lt  --name iih_base_lt_allihd_test  --dataset_root <dataset_dir> --dataset_name IHD --batch_size xx --init_port xxxx

• Apply pre-trained model

Download pre-trained model from Google Drive or BaiduCloud (access code: hqhw), and put latest_net_G.pth in the directory checkpoints/iih_base_lt_allihd. Run:

CUDA_VISIBLE_DEVICES=0 python test.py --model iih_base_lt  --name iih_base_lt_allihd  --dataset_root <dataset_dir> --dataset_name IHD --batch_size xx --init_port xxxx

Base Model with Guiding

• Train

CUDA_VISIBLE_DEVICES=0 python train.py --model iih_base_gd --name iih_base_gd_allihd_test --dataset_root <dataset_dir> --dataset_name IHD --batch_size xx --init_port xxxx

• Test

CUDA_VISIBLE_DEVICES=0 python test.py --model iih_base_gd --name iih_base_gd_allihd_test --dataset_root <dataset_dir> --dataset_name IHD --batch_size xx --init_port xxxx

• Apply pre-trained model

Download pre-trained model from Google Drive or BaiduCloud (access code: nqrc), and put latest_net_G.pth in the directory checkpoints/iih_base_gd_allihd. Run:

CUDA_VISIBLE_DEVICES=0 python test.py --model iih_base_gd --name iih_base_gd_allihd --dataset_root <dataset_dir> --dataset_name IHD --batch_size xx --init_port xxxx

Base Model with Lighting and Guiding

• Train

CUDA_VISIBLE_DEVICES=0 python train.py --model iih_base_lt_gd  --name iih_base_lt_gd_allihd_test  --dataset_root <dataset_dir> --dataset_name IHD --batch_size xx --init_port xxxx

• Test

CUDA_VISIBLE_DEVICES=0 python test.py --model iih_base_lt_gd  --name iih_base_lt_gd_allihd_test  --dataset_root <dataset_dir> --dataset_name IHD --batch_size xx --init_port xxxx

• Apply pre-trained model

Download pre-trained model from Google Drive or BaiduCloud (access code: kmgp), and put latest_net_G.pth in the directory checkpoints/iih_base_lt_gd_allihd. Run:

CUDA_VISIBLE_DEVICES=0 python test.py --model iih_base_lt_gd  --name iih_base_lt_gd_allihd  --dataset_root <dataset_dir> --dataset_name IHD --batch_size xx --init_port xxxx

Base Model with Lighting and Guiding on iHarmony4 and HVIDIT Datasets

• Train

CUDA_VISIBLE_DEVICES=0 python train.py --model iih_base_lt_gd  --name iih_base_lt_gd_newihd_test  --dataset_root <dataset_dir> --dataset_name newIHD --batch_size xx --init_port xxxx

• Test

CUDA_VISIBLE_DEVICES=0 python test.py --model iih_base_lt_gd  --name iih_base_lt_gd_newihd_test  --dataset_root <dataset_dir> --dataset_name newIHD --batch_size xx --init_port xxxx

• Apply pre-trained model

Download pre-trained model from Google Drive or BaiduCloud (access code: jnhg), and put latest_net_G.pth in the directory checkpoints/iih_base_lt_gd_allihd. Run:

CUDA_VISIBLE_DEVICES=0 python test.py --model iih_base_lt_gd  --name iih_base_lt_gd_newihd  --dataset_root <dataset_dir> --dataset_name newIHD --batch_size xx --init_port xxxx

Evaluation

We provide the code in ih_evaluation.py. Run:

# iHarmony4 dataset
CUDA_VISIBLE_DEVICES=0 python evaluation/ih_evaluation.py --dataroot <dataset_dir> --result_root  results/experiment/test_latest/images/ --evaluation_type our --dataset_name ALL
# iHarmony4 and HVIDIT datasets
CUDA_VISIBLE_DEVICES=0 python evaluation/ih_evaluation.py --dataroot <dataset_dir> --result_root  results/experiment/test_latest/images/ --evaluation_type our --dataset_name newALL

Quantitative Result

Real composite image harmonnization

More compared results can be found at Google Drive or BaduCloud (access code: lgs2).

Bibtex

If you use this code for your research, please cite our papers.

@InProceedings{Guo_2021_CVPR,
    author    = {Guo, Zonghui and Zheng, Haiyong and Jiang, Yufeng and Gu, Zhaorui and Zheng, Bing},
    title     = {Intrinsic Image Harmonization},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2021},
    pages     = {16367-16376}
}

Acknowledgement

For some of the data modules and model functions used in this source code, we need to acknowledge the repo of DoveNet and CycleGAN.