Awesome
[ECCV'20] CA-NoiseGAN
Project | Paper | Abstract | Long Video
Overview
CA-NoiseGAN is a PyTorch implementation of
"Learning Camera-Aware Noise Models",
Ke-Chi Chang, Ren Wang, Hung-Jin Lin, Yu-Lun Liu, Chia-Ping Chen, Yu-Lin Chang, Hwann-Tzong Chen
in European Conference on Computer Vision (ECCV) 2020 conference.
Modeling imaging sensor noise is a fundamental problem for image processing and computer vision applications. While most previous works adopt statistical noise models, real-world noise is far more complicated and beyond what these models can describe. To tackle this issue, we propose a data-driven approach, where a generative noise model is learned from real-world noise. The proposed noise model is camera-aware, that is, different noise characteristics of different camera sensors can be learned simultaneously, and a single learned noise model can generate different noise for different camera sensors. Experimental results show that our method quantitatively and qualitatively outperforms existing statistical noise models and learning-based methods.
<img src='imgs/results.jpg' width='100%' />Requirements
This test code is implemented under Python3.
Following libraries are required:
- PyTorch == 1.1.0
- see
requirements.txt
for more detail
If you want to visualize the results of Noise Flow, the libraries are also required:
- TensorFlow == 1.13.0
- tensorflow-probability >= 0.5.0con
Usage
-
Prepare Data
We prepare our test data asData.zip
in [Google Drive] and they are totally derived from SIDD dataset. Please unzip it and you can change thedata_dir
inconfig.yml
into your data path. -
Download Pretrained Models
We provide pretrained baseline models of noise models and denoisers ascheckpoints.zip
in [Google Dirve].
Please unzip it under the root directory. -
Prepare Runtime Environment
pip install -r requirements.txt
-
Test the Noise Models and Denoisers
You need to check the correctness of each path inconfig.yml
.
Moreover, you can modify the amount of samples and patch size. See config.yml for more detail. Then, you can run the following scripts.-
Noise Models
python test_noise_models.py --config config.yml
-
Denoisers
python test_denoisers.py --config config.yml
-
-
Visual Results
The results will be saved in./samples/
, including raw images and sRGB images. We provide the metrices we used in our paper, you can evaluate the quantitative results of each model, too.
Citation
@InProceedings{chang2020learning,
author={Chang, Ke-Chi and Wang, Ren and Lin, Hung-Jin and Liu, Yu-Lun and Chen, Chia-Ping and Chang, Yu-Lin and Chen, Hwann-Tzong},
title={Learning Camera-Aware Noise Models},
booktitle={Proceedings of European Conference on Computer Vision (ECCV)},
year={2020}
}
Resources
- SIDD Dataset
- Noise Flow (Tensorflow)
- Simple Camera Pipeline (Python, MATLAB)