Awesome
DMENet: Deep Defocus Map Estimation Network<br><sub>Official Implementation of the CVPR 2021 Paper</sub><br><sub>Project | Paper | Supp | Poster</sub><br><sub><sub></sub></sub>
This repository contains the official matlab implementation of SYNDOF generation used in the following paper:
<p align="left"> <a href="https://junyonglee.me/projects/DMENet"> <img width=85% src="./assets/teaser_DMENet.gif" /> </a> </p>Deep Defocus Map Estimation using Domain Adaptation<br> Junyong Lee<sup>1</sup>, Sungkil Lee<sup>2</sup>, Sunghyun Cho<sup>3</sup>, and Seungyong Lee<sup>1</sup><br> <sup>1</sup>POSTECH, <sup>2</sup>Sungkyunkwan University, <sup>3</sup>DGIST<br> IEEE Computer Vision and Pattern Recognition (CVPR) 2019<br>
Getting Started
Prerequisites
Tested environment
-
Setup environment
-
Option 1. install from scratch
$ git clone https://github.com/codeslake/DMENet.git $ cd DMENet # for CUDA10 $ conda create -y --name DMENet python=3.6 && conda activate DMENet $ sh install_CUDA10.0.sh # for CUDA11 (the name of conda environment matters) $ conda create -y --name DMENet_CUDA11 python=3.6 && conda activate DMENet_CUDA11 $ sh install_CUDA11.1.sh
-
Option 2. docker
$ nvidia-docker run --privileged --gpus=all -it --name DMENet --rm codeslake/dmenet:CVPR2019 /bin/zsh $ git clone https://github.com/codeslake/DMENet.git $ cd DMENet # for CUDA10 $ conda activate DMENet # for CUDA11 $ conda activate DMENet_CUDA11
-
-
Download and unzip datasets (OneDrive | Dropbox) under
[DATASET_ROOT]
.[DATASET_ROOT] ├── train │ ├── SYNDOF │ ├── CUHK │ └── Flickr └── test ├── CUHK ├── RTF └── SYNDOF
Note:
[DATASET_ROOT]
is currently set to./datasets/
. It can be specified by modifyingconfig.data_offset
in./config.py
.
-
Download pretrained weights of DMENet (OneDrive | Dropbox) and unzip it as in
[LOG_ROOT]/DMENet_CVPR2019/DMENet_BDCS/checkpoint/DMENet_BDCS.npz
([LOG_ROOT]
is currently set to./logs/
). -
Download pretrained VGG19 weights (OneDrive | Dropbox) and unzip as in
pretrained/vgg19.npy
(for training only).
Logs
-
Training and testing logs will be saved under
[LOG_ROOT]/DMENet_CVPR2019/[mode]/
:[LOG_ROOT] └──DMENet_CVPR2019 ├── [mode] │ ├── checkpoint # model checkpoint │ ├── log # scalar/image log for tensorboard │ ├── sample # sample images of training │ └── result # resulting images of evaluation └── ...
[LOG_ROOT]
can be modified withconfig.root_offset
in./config.py
.
Testing final model of CVPR 2019
Please note that due to the server issue, the checkpoint used for the paper is lost. <br/>The provided checkpoint is the new checkpoint that shows the closest evaluation results as in the paper.
Check out updated performance with the new checkpoint.
-
Test the final model by:
python main.py --mode DMENet_BDCS --test_set CUHK
Testing results will be saved in
[LOG_ROOT]/DMENet_CVPR2019/[mode]/result/[test_set]/
:... [LOG_ROOT]/DMENet_CVPR2019/[mode]/result/ └── [test_set] ├── image # input defocused images ├── defocus_map # defocus images (network's direct output in range [0, 1]) ├── defocus_map_min_max_norm # min-max normalized defocus images in range [0, 1] for visualization └── sigma_map_7_norm # sigma maps containing normalized standard deviations (in range [0, 1]) for a Gaussian kernel. For the actual standard deviation value, one should multiply 7 to this map.
Quantitative results are computed from matlab. (e.g., evaluation on the RTF dataset).
- Options
--mode
: The name of a model to test. The logging folder named with the[mode]
will be created as[LOG_ROOT]/DMENet_CVPR2019/[mode]/
. Default:DMENet_BDCS
--test_set
: The name of a dataset to evaluate.CUHK
|RTF0
|RTF1
|RTF1_6
|random
. Default:CUHK
- The folder structure can be modified in the function
get_eval_path(..)
in./config.py
. random
is for testing models with any images, which should be placed as[DATASET_ROOT]/test/random/*.[jpg|png]
.
- The folder structure can be modified in the function
- Options
-
Check out the evaluation code for the RTF dataset, and the deconvolution code.
Training & testing the network
-
Train the network by:
python main.py --is_train --mode [mode]
Note:
-
Test the network by:
python main.py --mode [mode] --test_set [test_set]
- arguments
--mode
: The name of a model to train. The logging folder named with the[mode]
will be created as[LOG_ROOT]/DMENet_CVPR2019/[mode]/
. Default:DMENet_BDCS
--is_pretrain
: Pretrain the network with the MSE loss (True
|False
). Default:False
--delete_log
: Deletes[LOG_ROOT]/DMENet_CVPR2019/[mode]/*
before training begins (True
|False
). Default:False
- arguments
Contact
Open an issue for any inquiries. You may also have contact with junyonglee@postech.ac.kr
Related Links
- CVPR 2021: Iterative Filter Adaptive Network for Single Image Defocus Deblurring [paper][code]
- ICCV 2021: Single Image Defocus Deblurring Using Kernel-Sharing Parallel Atrous Convolutions [paper][code]
- SYNDOF dataset generation repo [link]
License
<br> This software is being made available under the terms in the LICENSE file. Any exemptions to these terms require a license from the Pohang University of Science and Technology.
Citation
If you find this code useful, please consider citing:
@InProceedings{Lee2019DMENet,
author = {Junyong Lee and Sungkil Lee and Sunghyun Cho and Seungyong Lee},
title = {Deep Defocus Map Estimation Using Domain Adaptation},
booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2019}
}