Awesome

MC-Blur: A Comprehensive Benchmark for Image Deblurring

Our propsoed MC-Blur Benchmark

We construct a large-scale multi-cause (MC-Blur) dataset for image deblurring. It consists of four blur types: uniform blurs, motion blurs by averaging continuous frames, heavy defocus blurs, and real-world blurs. We collect these images from more than 1000 diverse scenes such as buildings, city scenes, vehicles, natural landscapes, people, animals, and sculptures. MC-Blur Benchmark consits of four different subsets, i.e., Real high-fps based Motion-blurred subset (RHM), large-kernel UHD Motion-blurred subset (UHDM), large-scale heavy defocus blurred subset (LSD), and Real Mixed Blurry Qualitative subset (RMBQ).

Downloads

The images of the dataset can be downloaded from the links below.

Google Drive

RHM-250-500-1000 (117G total data)
UHDM (278G total data)
LSD (34G total data)
RMBQ (110G total data)

Baidu Cloud (How to unzip?)

RHM-250-500-1000 (117G total data) (password:ohzp)
UHDM (278G total data) (password:p78n)
LSD (34G total data) (password:sbtu)
RMBQ (110G total data) (password:nwq8)

Download MC-Blur benchmark from the script, run

python download_data.py

Note: The above script will download all subsets of the MC-Blur. You can use "--data" to select. For example:

python download_data.py --data "UHDM_train_test"

Some visual examples of MC-Blur Dataset

Visual examples for each subset of our MC-Blur Dataset.

Some code steps in synthesizing dataset

See detail in README.

Benchmarking Study

Methods

Date	Publication	Title	Abbreviation	Code	Platform
2017	CVPR	Deep multi-scale convolutional neural network for dynamic scene deblurring paper	DeepDeblur	Code	Pytorch
2018	CVPR	Deblurgan: Blind motion deblurring using conditional adversarial networks paper	DeblurGAN	Code	Pytorch
2018	CVPR	Scale-recurrent network for deep image deblurring paper	SRN	Code	Tensorflow
2019	ICCV	DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better paper	DeblurGAN-v2	Code	Pytorch
2019	CVPR	Deep Stacked Hierarchical Multi-Patch Network for Image Deblurring paper	DMPHN	Code	Pytorch
2020	CVPR	Deblurring by Realistic Blurring paper	DBGAN	Code	Pytorch
2021	CVPR	Multi-Stage Progressive Image Restoration paper	MPRNet	Code	Pytorch
2022	CVPR	Restormer: Efficient Transformer for High-Resolution Image Restoration paper	Restormer	Code	Pytorch
2021	ICCV	Rethinking Coarse-To-Fine Approach in Single Image Deblurring paper	MIMO-UNet	Code	Pytorch

Metrics

Abbreviation	Full-/Non-Reference	Platform	Code
PSNR (Peak Signal-to-Noise Ratio)	Full-Reference
SSIM (Structural Similarity Index Measurement)	Full-Reference	MATLAB	Code
NIQE (Naturalness Image Quality Evaluator)	Non-Reference	MATLAB	Code
SSEQ (No-reference Image Quality Assessment Based on Spatial and Spectral Entropies)	Non-Reference	MATLAB	Code

Results for 250-fps images from RHM Set

Method	PSNR	SSIM	Parameter
DeepDeblur	30.38	0.8766	11.72 M
DeblurGAN	24.89	0.6364	6.07 M
SRN	30.57	0.8799	6.88 M
DeblurGAN-v2	26.99	0.8061	7.84 M
DMPHN	30.42	0.8768	21.69 M
DBGAN	27.89	0.8191	11.59 M
MPRNet	31.52	0.9239	20.13 M
Restormer	30.41	0.9106	26.10 M
MIMO-UNet	32.02	0.9285	6.81 M

Results for 500-fps images from RHM Set

Method	PSNR	SSIM	Parameter
DeepDeblur	31.08	0.8974	11.72 M
DeblurGAN	24.66	0.6748	6.07 M
SRN	31.54	0.9051	6.88 MB
DeblurGAN-v2	27.67	0.8320	7.84 M
DMPHN	31.43	0.9018	21.69 M
DBGAN	28.36	0.8388	11.59 M
MPRNet	32.08	0.9300	20.13 M
Restormer	30.98	0.9160	26.10 M
MIMO-UNet	32.89	0.9398	6.81 M

Results for 1000-fps images from RHM Set

Method	PSNR	SSIM	Parameter
DeepDeblur	32.41	0.8966	11.72 M
DeblurGAN	25.20	0.6535	6.07 M
SRN	32.69	0.0.9016	6.88 M
DeblurGAN-v2	29.81	0.8461	7.84 M
DMPHN	32.41	0.9096	21.69 M
DBGAN	29.66	0.8318	11.59 M
MPRNet	33.36	0.9332	20.13 M
Restormer	32.77	0.9264	26.10 M
MIMO-UNet	33.75	0.9360	6.81 M

Results on UHDM Set

Method	PSNR	SSIM	Parameter
DeepDeblur	22.23	0.6322	11.72 M
DeblurGAN	20.39	0.5568	6.07 M
SRN	22.28	0.6346	6.88 M
DeblurGAN-v2	21.03	0.5839	7.84 M
DMPHN	22.20	0.6378	21.69 M
DBGAN	21.52	0.6025	11.59 M
MPRNet	23.70	0.7472	20.13 M
Restormer	22.39	0.7356	26.10 M
MIMO-UNet	22.97	0.7317	6.81 M

Results on LSD Set

Method	PSNR	SSIM	Parameter
DeepDeblur	20.73	0.7218	11.72 M
DeblurGAN	20.04	0.6335	6.07 M
SRN	21.66	0.7664	6.88 M
DeblurGAN-v2	21.13	0.6964	7.84 M
DMPHN	21.23	0.7519	21.69 M
DBGAN	21.56	0.7536	11.59 M
MPRNet	21.32	0.7897	20.13 M
Restormer	22.35	0.8072	26.10 M
MIMO-UNet	22.56	0.7985	6.81 M

Citation

If you think this work is useful for your research, please cite the following paper.

@inproceedings{zhang2023benchmarking,
  title={MC-Blur: A Comprehensive Benchmark for Image Deblurring},
  author={Zhang, Kaihao and Wang, Tao and Luo, Wenhan and Chen, Boheng and Ren, Wenqi and Stenger, Bjorn and Liu, Wei and Li, Hongdong and Yang Ming-Hsuan},
  booktitle={IEEE Transactions on Circuits and Systems for Video Technology},
  year={2023}
}

License

The MC-Blur dataset is released under CC BY-NC-ND license.