Awesome

Single Image Deraining using a Recurrent Multi-scale Aggregation and Enhancement Network (ICME'19)

[pdf][code][DerainZoo]

Youzhao Yang, Hong Lu; School of Computer Science, Fudan University

Abstract

Single image deraining is an ill-posed inverse problem due to the presence of non-uniform rain shapes, directions, and densities in images. In this paper, we propose a novel progressive single image deraining method named Recurrent Multiscale Aggregation and Enhancement Network (ReMAEN). Differing from previous methods, ReMAEN contains a symmetric structure where recurrent blocks with shared channel attention are applied to select useful information collaboratively and remove rain streaks stage by stage. In ReMAEN, a Multi-scale Aggregation and Enhancement Block (MAEB) is constructed to detect multi-scale rain details. Moreover, to better leverage the rain details from rainy images, ReMAEN enables a symmetric skipping connection from low level to high level. Extensive experiments on synthetic and real-world datasets demonstrate that our method outperforms the state-of-the-art methods tremendously. Furthermore, ablation studies are conducted to show the improvements obtained by each module in ReMAEN.

Dataset

Synthetic Datasets

Real-World Datasets

Pre-trained Model

We note that these models is trained on NVIDIA GeForce GTX1080Ti:

Requirements

• python 3.6.5
• opencv 3.4.2
• numpy 1.14.3
• argparse 1.1
• tensorflow_gpu >=1.4.0 & < 2.0

Usages

• Clone this repo

   $ git clone https://github.com/nnUyi/ReMAEN
   $ cd ReMAEN
   $ cd codes

• Train

   $ CUDA_VISIBLE_DEVICES=0 python main.py --is_training True
                                           --train_dataset Rain100L
                                           --test_dataset Rain100L
                                           --trainset_size 200
                                           --testset_size 100
                                           --iterations 40000
                                           --lr 0.001
                                           --batch_size 32

• Test

   $ CUDA_VISIBLE_DEVICES=0 python main.py --is_testing True
                                           --train_dataset Rain100L
                                           --test_dataset Rain100L
                                           --trainset_size 200
                                           --testset_size 100
                                           --batch_size 32

• Where should your datasets place?
  • In the 32rd and 33rd lines of settings.py, you can see the --data_path and --test_path settings. You should download the datasets (or putting your own dataset) in the target path. And you should add your dataset format in train_dic & test_dic in the 50th and 59th lines of settings.py.

Results

Recurrent rain removal analysis

Edge loss analysis

Rainy	w/o edge loss	with edge loss	ground truth

Average PSNR and SSIM values on five synthetic datasets

References

[1] Wenhan Yang, Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan, “Deep joint rain detection and removal from a single image,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1357–1366.

[2] He Zhang and Vishal M. Patel, “Density-aware single image deraining using a multi-stream dense network,” in The IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 695–704.

[3] Jinshan Pan, Sifei Liu, Deqing Sun, Jiawei Zhang, Yang Liu, Jimmy Ren, Zechao Li, Jinhui Tang, Huchuan Lu, and Yu-Wing Tai, “Learning dual convolutional neural networks for low-level vision,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 3070–3079.

[4] Xia Li, Jianlong Wu, Zhouchen Lin, Hong Liu, and Hongbin Zha, “Recurrent squeeze-and-excitation context aggregation net for single image deraining,” in The European Conference on Computer Vision (ECCV), 2018, pp. 262–277.

Acknowledgements

• This work was supported in part by National Natural Science Foundation of China (No. U1509206).

Citation

If you find the resource usefully, please cite the following:

   @inproceedings{yang2019single,
      title={Single Image Deraining using a Recurrent Multi-scale Aggregation and Enhancement Network},
      author={Yang, Youzhao and Lu, Hong},   
      booktitle={IEEE International Conference on Multimedia and Expo (ICME)},
      year={2019}
   }