Home

Awesome

<div align="center"> <a href="https://whyy.site/paper/lcdp"> <img src="imgs/title.webp"/> </a> <a href="https://paperswithcode.com/sota/image-enhancement-on-exposure-errors?p=local-color-distributions-prior-for-image"> <img src="https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/local-color-distributions-prior-for-image/image-enhancement-on-exposure-errors" alt="PWC" /> </a>

๐ŸŒ Website ย ยทย  ๐Ÿ“ƒ Paper ย ยทย  ๐Ÿ—ƒ๏ธ Dataset

</div>

Abstract: Existing image enhancement methods are typically designed to address either the over- or under-exposure problem in the input image. When the illumination of the input image contains both over- and under-exposure problems, these existing methods may not work well. We observe from the image statistics that the local color distributions (LCDs) of an image suffering from both problems tend to vary across different regions of the image, depending on the local illuminations. Based on this observation, we propose in this paper to exploit these LCDs as a prior for locating and enhancing the two types of regions (i.e., over-/under-exposed regions). First, we leverage the LCDs to represent these regions, and propose a novel local color distribution embedded (LCDE) module to formulate LCDs in multi-scales to model the correlations across different regions. Second, we propose a dual-illumination learning mechanism to enhance the two types of regions. Third, we construct a new dataset to facilitate the learning process, by following the camera image signal processing (ISP) pipeline to render standard RGB images with both under-/over-exposures from raw data. Extensive experiments demonstrate that the proposed method outperforms existing state-of-the-art methods quantitatively and qualitatively.

๐Ÿ“ป News

๐Ÿ”ฅ Our Model

Our model

โš™๏ธ Setup

  1. Clone git clone https://github.com/onpix/LCDPNet.git
  2. Go to directory cd LCDPNet
  3. Install required packages pip install -r requirements.txt

We also provide env.yaml for quickly installing packages. Note that you may need to modify the env name to prevent overwriting your existing enviroment, or modify cudatoolkit and cudnn version in env.yaml to match your local cuda version.

โŒจ๏ธ How to run

To train our model:

  1. Prepare data: Modify src/config/ds/train.yaml and src/config/ds/valid.yaml.
  2. Modify configs in src/config. Note that we use hydra for config management.
  3. Run: python src/train.py name=<experiment_name> num_epoch=200 log_every=2000 valid_every=20

To test our model:

  1. Prepare data: Modify src/config/ds/test.yaml
  2. Run: python src/test.py checkpoint_path=<file_path>

๐Ÿ“‚ Dataset & Pretrained Model

The LCDP Dataset is here: [Google drive]. Please unzip lcdp_dataset.7z. The training and test images are:

TrainTest
Inputinput/*.pngtest-input/*.png
GTgt/*.pngtest-gt/*.png

We provide the two pretrained models: pretrained_models/trained_on_ours.ckpt and pretrained_models/trained_on_MSEC.ckpt for researchers to reproduce the results in Table 1 and Table 2 in our paper. Note that we train pretrained_models/trained_on_MSEC.ckpt on the Expert C subset of the MSEC dataset with both over and under-exposed images.

FilenameTraining dataTesting dataTest PSNRTest SSIM
trained_on_ours.ckptOursOur testing data23.2390.842
trained_on_MSEC.ckptMSECMSEC testing data (Expert C)22.2950.855

Our model is lightweight. Experiments show that increasing model size will further improve the quality of the results. To train a bigger model, increase the values in runtime.bilateral_upsample_net.hist_unet.channel_nums.

๐Ÿ”— Cite This Paper

If you find our work or code helpful, or your research benefits from this repo, please cite our paper:

@inproceedings{wang2022lcdp,
    title =        {Local Color Distributions Prior for Image Enhancement},
    author =       {Haoyuan Wang, Ke Xu, and Rynson W.H. Lau},
    booktitle =    {Proceedings of the European Conference on Computer Vision (ECCV)},
    year =         {2022}
}