Awesome
Denoising Diffusion Post-Processing for Low-Light Image Enhancement
News
- The article is officially published in Pattern Recognition and is openly accessible [here] 🎉
Abstract: Low-light image enhancement (LLIE) techniques attempt to increase the visibility of images captured in low-light scenarios. However, as a result of enhancement, a variety of image degradations such as noise and color bias are revealed. Furthermore, each particular LLIE approach may introduce a different form of flaw within its enhanced results. To combat these image degradations, post-processing denoisers have widely been used, which often yield oversmoothed results lacking detail. We propose using a diffusion model as a post-processing approach, and we introduce Low-light Post-processing Diffusion Model (LPDM) in order to model the conditional distribution between under-exposed and normally-exposed images. We apply LPDM in a manner which avoids the computationally expensive generative reverse process of typical diffusion models, and post-process images in one pass through LPDM. Extensive experiments demonstrate that our approach outperforms competing post-processing denoisers by increasing the perceptual quality of enhanced low-light images on a variety of challenging low-light datasets. Source code is available at https://github.com/savvaki/LPDM.
Download Results
The image results for all techniques and datasets are available for download here.
undarken
directories contain $\hat{\boldsymbol{x}}_0^\eta$ for each low-light enhancement technique $\eta$. In other words, these are the results before post-processing.denoised
directories contain the results of LPDM in thelpdm_lol
subdirectory, as well as the results for the ablation studies. For the LOL dataset, the results of NAFNet and BM3D are also provided. BM3D results are provided for different values of $\sigma$.- The LPDM results have subdirectories in the format phi_s which indicate the $\phi$ and $s$ parameters used.
Requirements
Set up and activate the virtual conda environment:
conda env create -f environment.yml
conda activate lpdm
Test
- Download the pre-trained model
.ckpt
file and.yaml
file here and place them in thecheckpoints
directory:───checkpoints ├───lpdm_lol.ckpt └───lpdm_lol.yaml
- Modify the
s
andphi
(φ) parameters inconfigs/test/denoise.yaml
as desired. - Place your low-light images (c) in the
test/dark
directory - Place your enhanced images by any enhancer η in the
test/eta
directory. Make sure the images have the same names as the files intest/dark
- Run the following commands:
cd scripts python denoise_config.py
- Denoised results can be found in
test/denoised
- Run the following command for info about optional extra settings:
python denoise_config.py --help
. - By default, the device is set to CPU. To use GPU pass the argument
--device "cuda"
Calculate Metrics
- Modify
configs/test/metrics.yaml
and specify the denoised image glob pathpred_path
and the ground truth glob pathtarget_path
. To use non-reference metrics, removetarget_path
from the config. - Run the following commands:
cd scripts python calculate_metrics.py
- Metrics will be saved in
test/results.csv
. - For additional options run
python calculate_metrics.py --help
Train
-
Download the LOL Dataset here. Unzip the dataset and place the contents in the
datasets/lol
directory:───datasets └───lol ├───eval15 │ ├───high │ └───low └───our485 ├───high └───low
-
For the commands below, PyTorch lightning will create a
logs
directory where checkpoints will be saved. -
For GPU training options pass the
--gpu
argument according to PyTorch Lighting documentation. The examples below use one GPU. -
Train and log locally:
python main.py --base configs/train/lpdm_lol.yaml --gpu 0,
-
Train and log metrics with wandb:
- Set your wandb environment variables:
WANDB_API_KEY
,WANDB_ENTITY
,WANDB_PROJECT
python main.py --base configs/train/lpdm_lol.yaml --logger_to_use wandb --gpu 0,
- Set your wandb environment variables:
-
An example command to resume a saved checkpoint in the directory
RUN_DIRECTORY_NAME_HERE
using wandb logging:python main.py --resume logs/RUN_DIRECTORY_NAME_HERE --logger_to_use wandb --gpu 0,
-
To reduce the size of the model, reduce the
model_channels
parameter inconfigs/train/lpdm_lol.yaml
BibTeX Citation
@article{panagiotou2024denoising,
title = {Denoising diffusion post-processing for low-light image enhancement},
journal = {Pattern Recognition},
volume = {156},
pages = {110799},
year = {2024},
issn = {0031-3203},
doi = {https://doi.org/10.1016/j.patcog.2024.110799},
url = {https://www.sciencedirect.com/science/article/pii/S0031320324005508},
author = {Savvas Panagiotou and Anna S. Bosman}
}
References
This repository is a derivative of the original Stable Diffusion repository.