Awesome

Single Image Deraining via Recurrent Hierarchy Enhancement Network (ACM'MM19)

[pdf][code][DerainZoo]

Youzhao Yang, and Hong Lu; School of Computer Science, Fudan University

Abstract

Single image deraining is an important problem in many computer vision tasks since rain streaks can severely hamper and degrade the visibility of images. In this paper, we propose a novel network named Recurrent Hierarchy Enhancement Network (ReHEN) to remove rain streaks from rainy images stage by stage. Unlike previous deep convolutional network methods, we adopt a Hierarchy Enhancement Unit (HEU) to fully extract local hierarchical features and generate effective features. Then a Recurrent Enhancement Unit (REU) is added to keep the useful information from HEU and benefit the rain removal in the later stages. To focus on different scales, shapes, and densities of rain streaks adaptively, Squeeze-and-Excitation (SE) block is applied in both HEU and REU to assign different scale factors to high-level features. Experiments on five synthetic datasets and a real-world rainy image set show that the proposed method outperforms the state- of-the-art methods considerably. The source code is available at https://github.com/nnUyi/ReHEN .

Dataset

Synthetic Datasets

Real-World Datasets

Pre-trained Model

We note that these models is trained on NVIDIA GeForce GTX1070:

Requirements

• python 3.6.5
• opencv 3.4.2
• numpy 1.14.3
• argparse 1.1
• tensorflow_gpu >=1.4.0 & < 2.0

Usages

• Clone this repo

   $ git clone https://github.com/nnUyi/ReHEN
   $ cd ReHEN
   $ cd codes

• Train

   $ CUDA_VISIBLE_DEVICES=0 python main.py --is_training True
                                           --train_dataset Rain100L
                                           --test_dataset Rain100L
                                           --trainset_size 200
                                           --testset_size 100
                                           --iterations 60000
                                           --lr 0.005
                                           --batch_size 32

• Test

   $ CUDA_VISIBLE_DEVICES=0 python main.py --is_testing True
                                           --train_dataset Rain100L
                                           --test_dataset Rain100L
                                           --trainset_size 200
                                           --testset_size 100
                                           --batch_size 32

• Where should your datasets place?
  • In the 34rd and 35rd lines of settings.py, you can see the --data_path and --test_path settings. You should download the datasets (or putting your own dataset) in the target path. And you should add your dataset format in train_dic & test_dic in the 50th and 59th lines of settings.py.

Results

Visualization of Hidden Statuses

Average PSNR and SSIM values on five synthetic datasets

References

[1] Wenhan Yang, Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan, “Deep joint rain detection and removal from a single image,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1357–1366.

[2] He Zhang and Vishal M. Patel, “Density-aware single image deraining using a multi-stream dense network,” in The IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 695–704.

[3] Jinshan Pan, Sifei Liu, Deqing Sun, Jiawei Zhang, Yang Liu, Jimmy Ren, Zechao Li, Jinhui Tang, Huchuan Lu, and Yu-Wing Tai, “Learning dual convolutional neural networks for low-level vision,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 3070–3079.

[4] Xia Li, Jianlong Wu, Zhouchen Lin, Hong Liu, and Hongbin Zha, “Recurrent squeeze-and-excitation context aggregation net for single image deraining,” in The European Conference on Computer Vision (ECCV), 2018, pp. 262–277.

Acknowledgements

• This work was supported in part by National Natural Science Foundation of China (No. U1509206).

Citation

If you find the resource useful, please cite the following:

   @inproceedings{yang2019single,
      title={Single Image Deraining via Recurrent Hierarchy Enhancement Network},
      author={Yang, Youzhao and Lu, Hong},   
      booktitle={ACM International Conference on Multimedia (ACM'MM)},
      year={2019}
   }