Home

Awesome

Realistic Blur Synthesis for Learning Image Deblurring

Project | Paper | Supple

Official Implementation of ECCV Paper

Realistic Blur Synthesis for Learning Image Deblurring<br> Jaesung Rim, Geonung Kim, Jungeon Kim, Junyong Lee, Seungyong Lee, Sunghyun Cho. <br> POSTECH<br> IEEE European Conference on Computer Vision (ECCV) 2022<br>

Pytorch implementation

We provide simple Dataset modules for adopting our pipeline. This is slightly different from the tensorflow implementation.

Please refer to Uformer-RSBlur.

# ./Uformer-RSBlur/dataset/dataset_RealisticDeblur.py

class RealisticGoProABMEDataset(Dataset):
    def __init__(self, image_dir, patch_size=256, image_aug=True, realistic_pipeline=True):
        ...

    def __len__(self):
        return len(self.image_list)

    def __getitem__(self, idx):
        ...

class RealisticGoProUDataset(Dataset):
    def __init__(self, image_dir, patch_size=256, image_aug=True, realistic_pipeline=True):
        ...

    def __len__(self):
        return len(self.image_list)

    def __getitem__(self, idx):
        ...

Results with the proposed method.

<img src="./imgs/qualatitive_results.png" width="100%" alt="Real Photo"> <details> <summary><strong>Results of analysis</strong> (click) </summary>
CRFInterp.Sat.NoiseISPPSNR / SSIM
Linear30.12 / 0.7727
sRGB30.90 / 0.7805
sRGB30.20 / 0.7468
sRGBG31.77 / 0.8275
sRGBOursG31.83 / 0.8265
sRGBOursG+P32.06 / 0.8322
</details>

Installation

git clone https://github.com/rimchang/RSBlur.git

Tested environment

We recommend a virtual environment using conda or docker.

SRN-Deblur

Download

Dataset [Google Drive] [Postech]

<details> <summary><strong>Descriptions</strong> (click) </summary>

RSBlur

# RSBlur.zip

RSBlur
├── 0001
│   ├── 000001
│   │   ├── real_blur/real_blur.png # real blurred image
│   │   ├── avg65_img/avg_blur.png # synthetic blurred image using frame interpolation
│   │   ├── avg65_mask_100/avg_blur.png # saturation mask
│   │   ├── gt/gt_sharp.png # ground truth sharp image
...

GoPro_INTER_ABME

# GoPro_INTER_ABME.zip

GoPro_INTER_ABME
├── GOPR0372_07_00 
│   ├── 000001
│   │   ├── avg_inter_img/avg_blur.png # synthetic blurred image using frame interpolation
│   │   ├── avg_inter_mask_100/avg_blur.png # saturation mask
│   │   ├── gt/gt_sharp.png # ground truth sharp image
...

GoPro_U

# GoPro_U.zip

GoPro_U
├── centroid_blurred_img
│   ├── 0_GOPR0372_07_00_000047_003754_kernel_25_blurred.png # synthetic blurred image using a synthetic blur kernel
│   ...
├── centroid_blurred_mask_100
│   ├── 0_GOPR0372_07_00_000047_003754_kernel_25_blurred.png # saturation mask
│   ...
├── target_img
│   ├── 0_GOPR0372_07_00_000047_003754_kernel_25_gt.png # ground truth sharp image
│   ...
</details>

Dataset splits [link]

Pre-trained models [Google Drive] [Postech]

<details> <summary><strong>Descriptions</strong> (click) </summary> </details>

Training

# ./SRN-Deblur-RSBlur
# All datasets should be located in SRN-Deblur-RSBlur/dataset

# RSBlur
python run_model.py --phase=train --checkpoint_path=0719_SRN-Deblur_RSBlur_real --sat_synthesis=None --noise_synthesis=None --datalist=../datalist/RSBlur/RSBlur_real_train.txt --gpu=0
python run_model.py --phase=train --checkpoint_path=0719_SRN-Deblur_RSBlur_syn_with_ours --sat_synthesis=sat_synthesis --noise_synthesis=poisson_RSBlur --cam_params_RSBlur=1 --datalist=../datalist/RSBlur/RSBlur_syn_train.txt --gpu=0

# GoPro_INTER_ABME
python run_model.py --phase=train --checkpoint_path=0719_SRN-Deblur_GoPro_ABME_with_ours --target_dataset=RealBlur --sat_synthesis=sat_synthesis --noise_synthesis=poisson_gamma --cam_params_RealBlur=1 --adopt_crf_realblur=1 --datalist=../datalist/GoPro/GoPro_INTER_ABME_train.txt --gpu=0

# GoPro_U
python run_model.py --phase=train --checkpoint_path=0719_SRN-Deblur_U_with_ours --target_dataset=RealBlur --sat_synthesis=sat_synthesis --noise_synthesis=poisson_gamma --cam_params_RealBlur=1 --adopt_crf_realblur=1 --datalist=../datalist/GoPro/GoPro_U_train.txt --gpu=0

Testing

# ./SRN-Deblur-RSBlur
# All datasets should be located in SRN-Deblur-RSBlur/dataset

# RSBlur
python run_model.py --phase=test --checkpoint_path=SRN-Deblur_RSBlur_real --datalist=../datalist/RSBlur/RSBlur_real_test.txt --gpu=0
python run_model.py --phase=test --checkpoint_path=SRN-Deblur_RSBlur_syn_with_ours --datalist=../datalist/RSBlur/RSBlur_real_test.txt --gpu=0

# RealBlur
python run_model.py --phase=test --checkpoint_path=SRN-Deblur_GoPro_ABME_with_ours --datalist=../datalist/RealBlur_J_test_list.txt --gpu=0
python run_model.py --phase=test --checkpoint_path=SRN-Deblur_GoPro_U_with_ours --datalist=../datalist/RealBlur_J_test_list.txt --gpu=0

Evaluation

# ./evaluation

python evaluate_RSBlur.py --input_dir=../SRN-Deblur-RSBlur/testing_res/SRN-Deblur_RSBlur_real --gt_root=../SRN-Deblur-RSBlur/dataset/RSBlur;
python evaluate_RealBlur.py --input_dir=../SRN-Deblur-RSBlur/testing_res/SRN-Deblur_U_with_ours --gt_root=../SRN-Deblur-RSBlur/dataset/RealBlur-J_ECC_IMCORR_centroid_itensity_ref;

Real-world Deblurring Benchmark

We provide an additional deblurring benchmark to provide the basis for future deblurring research. All below models are trained on real blurred images of the RSBlur training set.

<img src="./imgs/benchmark_for_web.png" width="100%" alt="Real Photo"> <details> <summary><strong>Results of benchmark</strong> (click) </summary>
MethodsPSNR / SSIMLink
Uformer-B33.98 / 0.8660Result / Weight / Test Code
Restormer33.69 / 0.8628Result / Weight
MPRNet33.61 / 0.8614Result / Weight
MiMO-UNet+33.37 / 0.8560Result / Weight
MiMO-UNet32.73 / 0.8457Result / Weight
SRN-Deblur32.53 / 0.8398Result / Weight
</details>

License

The RSBlur dataset is released under CC BY 4.0 license.

Acknowledment

The code is based on SRN-Deblur, CBDNet and UID.

Citation

If you use our dataset for your research, please cite our paper.

@inproceedings{rim_2022_ECCV,
 title={Realistic Blur Synthesis for Learning Image Deblurring},
 author={Rim, Jaesung and Kim, Geonung and Kim, Jungeon and Lee, Junyong and Lee, Seungyong and Cho, Sunghyun},
 booktitle={Proceedings of the European Conference on Computer Vision (ECCV)},
 year={2022}
}