Home

Awesome

Learning to See in the Dark in PyTorch.

This repo implements Learning to See in the Dark models presented in [1].

Test(Inference)

Usage:

python demo.py test <options>

In order to use pretrained models by authors of [1], which are trained using TensorFlow, specify a converted pretrained model file(see here) to --tf_weight_file.

If you use trained models by this repo, specify model files to --resume option.

Training

Usage:

python demo.py train <options>

Options

Pretrained models

The following weight files are models converted from authors' TensorFlow models listed in the first two rows, and models trained from scratch by codes in this repo(third row). I converted the authors TensorFlow checkpoint files to pickle format to load their weight values into PyTorch models.

cameradownload linkmemo
Sonylinkconverted from TensorFlow model
Fujilinkconverted from TensorFlow model
Sonylinktrained model using this repo from scratch
Fuji-not trained by this repo

Dataset

To download datasets by Sony and Fuji cameras, see authors' site.

PSNR/SSIM results

Table below shows PSNR and SSIM results using converted authors' TensorFlow models and a trained model by codes in this repo. Two types of test data are used:

modeltest dataSony(PSNR/SSIM(#images))Fuji(PSNR/SSIM(#images))
paper[1]-28.88/0.78726.61/0.680
converted from TensorFlowall28.88/0.7980(598)27.01/0.7241(524)
converted from TensorFlow00 only28.59/0.7831(93)26.61/0.7040(94)
trained from scratchall28.55/0.7878(598)-
trained from scratch00 only28.25/0.7721(93)-

Some Results

Sony

Camera: Sony a7S II

Info: ISO-2000 f/9 1/10s

Ground truth(RAW file: Sony/long/10106_00_30s.ARW)

ground_truth

The following images are inference and white balance results for RAW file: Sony/short/10106_00_0.1s.ARW

inference by the converted TensorFlow model

tensorflow

inference by the PyTorch model trained from scratch

pytorch

white balance by rawpy(raw.postprocess(use_camera_wb=True, half_size=False, no_auto_bright=False, output_bps=16))

rawpy

white balnce by Imagemagick convert command(e.g. convert arw:10106_00_0.1s.ARW 10106_00_0.1s.png)

imagemagick

Fuji

Camera: Fuji X-T2

Info: ISO-800 f/7.1 1/30s

Ground truth(RAW file: Fuji/long/10068_00_10s.RAF)

ground_truth

The following images are inference and white balance results for RAW file: Fuji/short/10068_00_0.033s.RAF

inference by the converted TensorFlow model

tensorflow

white balance by rawpy(raw.postprocess(use_camera_wb=True, half_size=False, no_auto_bright=False, output_bps=16))

rawpy

white balnce by Imagemagick convert command(e.g. convert raf:10068_00_0.033s.RAF 10068_00_0.033s.png)

imagemagick

References

  1. Chen, C., Chen, Q., Xu, J., & Koltun, V. (2018). Learning to See in the Dark. arXiv preprint arXiv:1805.01934.
    arXiv, github, Project Website