Home

Awesome

Low-Light Image and Video Enhancement Using Deep Learning: A Survey

teaser

This repository provides:

1) a unified online platform, LLIE-Platform http://mc.nankai.edu.cn/ll/, that covers many popular deep learning-based LLIE methods, of which the results can be produced through a user-friendly web interface, contains a low-light image and video dataset.

2) a new dataset LLIV-Phone https://drive.google.com/file/d/1QS4FgT5aTQNYy-eHZ_A89rLoZgx_iysR/view?usp=sharing, in which the images and videos are taken by various phones' cameras under diverse illumination conditions and scenes, and

3) collects deep learning-based low-light image and video enhancement methods, datasets, and evaluation metrics.

More content and details can be found in our Survey Paper: Low-Light Image and Video Enhancement Using Deep Learning: A Survey.

我们提供中文翻译版:基于深度学习的低照度图像与视频增强综述.

We provide the comparison results on the real low-light videos taken by different mobile phones’ cameras at YouTube https://www.youtube.com/watch?v=Elo9TkrG5Oo&t=6s.

✌We will periodically update the content. Welcome to let us know if we miss your work that is published in top-tier Journal or conference. We will add it.

✌Our LLIE-Platform supports the function of download. Please right click and then save the figure.

If you use this dataset or platform, please cite our paper. Please hit the star at the top-right corner. Thanks!

Please hit the star in the repo when you ask for any code. Thanks!

📣News

  1. The survey is accepted by TPAMI.

  2. We newly add the Zero-DCE++ to the LLIE-Platform. Have Fun!

Zero-DCE++: C. Li, C. Guo, and C. C. Loy, Learning to enhance low-light image via zero-reference deep curve estimation, TPAMI, 2021.

🌱Contents

  1. LLIE-Platform
  2. LLIV-Phone Dataset
  3. Methods
  4. Datasets
  5. Metrics
  6. Citation

📋LoLi-Platform

Currently, the LLIE-Platform covers 14 popular deep learning-based LLIE methods including LLNet, LightenNet, Retinex-Net, EnlightenGAN, MBLLEN, KinD, KinD++, TBEFN, DSLR, DRBN, ExCNet, Zero-DCE, Zero-DCE++, and RRDNet, where the results of any inputs can be produced through a user-friendly web interface. Have fun: LLIE-Platform.

📋LLIV-Phone

Overview LLIV-Phone dataset contains 120 videos (45,148 images) taken by 18 different phones' cameras including iPhone 6s, iPhone 7, iPhone7 Plus, iPhone8 Plus, iPhone 11, iPhone 11 Pro, iPhone XS, iPhone XR, iPhone SE, Xiaomi Mi 9, Xiaomi Mi Mix 3, Pixel 3, Pixel 4, Oppo R17, Vivo Nex, LG M322, OnePlus 5T, Huawei Mate 20 Pro under diverse illumination conditions (e.g., weak illumination, underexposure, dark, extremely dark, back-lit, non-uniform light, color light sources, etc.) in the indoor and outdoor scenes.

Anyone can access the LLIV-Phone dataset via

Google Drive: https://drive.google.com/file/d/1QS4FgT5aTQNYy-eHZ_A89rLoZgx_iysR/view?usp=sharing or

Baidu Cloud:https://pan.baidu.com/s/1-8PF3dfbtlHlmk9y5ZKx_w, Password: s0b9)

📋Methods

Overview

DatePublicationTitleAbbreviationCodePlatform
2017PRLLNet: A deep autoencoder approach to natural low-light image enhancement paperLLNetCodeTheano
2018PRLLightenNet: A convolutional neural network for weakly illuminated image enhancement paperLightenNetCodeCaffe & MATLAB
2018BMVCDeep retinex decomposition for low-light enhancement paperRetinex-NetCodeTensorFlow
2018BMVCMBLLEN: Low-light image/video enhancement using CNNs paperMBLLENCodeTensorFlow
2018TIPLearning a deep single image contrast enhancer from multi-exposure images paperSCIECodeCaffe & MATLAB
2018CVPRLearning to see in the dark paperChen et al.CodeTensorFlow
2018NeurIPSDeepExposure: Learning to expose photos with asynchronously reinforced adversarial learning paperDeepExposureTensorFlow
2019ICCVSeeing motion in the dark paperChen et al.CodeTensorFlow
2019ICCVLearning to see moving object in the dark paperJiang and ZhengCodeTensorFlow
2019CVPRUnderexposed photo enhancement using deep illumination estimation paperDeepUPECodeTensorFlow
2019ACMMMKindling the darkness: A practical low-light image enhancer paperKinDCodeTensorFlow
2019ACMMM (IJCV)Kindling the darkness: A practical low-light image enhancer paper (Beyond brightening low-light images paper)KinD (KinD++)CodeTensorFlow
2019ACMMMProgressive retinex: Mutually reinforced illumination-noise perception network for low-light image enhancement paperWang et al.Caffe
2019TIPLow-light image enhancement via a deep hybrid network paperRen et al.Caffe
2019(2021)arXiv(TIP)EnlightenGAN: Deep light enhancement without paired supervision paper arxivEnlightenGANCodePyTorch
2019ACMMMZero-shot restoration of back-lit images using deep internal learning paperExCNetCodePyTorch
2020CVPRZero-reference deep curve estimation for low-light image enhancement paperZero-DCECodePyTorch
2020CVPRFrom fidelity to perceptual quality: A semi-supervised approach for low-light image enhancement paperDRBNCodePyTorch
2020ACMMMFast enhancement for non-uniform illumination images using light-weight CNNs paperLv et al.TensorFlow
2020ACMMMIntegrating semantic segmentation and retinex model for low light image enhancement paperFan et al.
2020CVPRLearning to restore low-light images via decomposition-and-enhancement paperXu et al.CodePyTorch
2020AAAIEEMEFN: Low-light image enhancement via edge-enhanced multi-exposure fusion network paperEEMEFNPyTorch
2020TIPLightening network for low-light image enhancement paperDLNPyTorch
2020TMMLuminance-aware pyramid network for low-light image enhancement paperLPNetPyTorch
2020ECCVLow light video enhancement using synthetic data produced with an intermediate domain mapping paperSIDGANTensorFlow
2020TMMTBEFN: A two-branch exposure-fusion network for low-light image enhancement paperTBEFNCodeTensorFlow
2020ICMEZero-shot restoration of underexposed images via robust retinex decomposition paperRRDNetCodePyTorch
2020TMMDSLR: Deep stacked laplacian restorer for low-light image enhancement paperDSLRCodePyTorch
2021TPAMILearning to enhance low-light image via zero-reference deep curve estimation paperZero-DCE++CodePyTorch
2021CVPRRetinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement paperRUASCodePyTorch
2021CVPRLearning temporal consistency for low light video enhancement from single images paperZhang et al.CodePyTorch
2021CVPRNighttime visibility enhancement by increasing the dynamic range and suppression of light effects paperSharma and Tan
2021TCSVTRetinexDIP: A unified deep framework for low-light image enhancement paperRetinexDIPCodePyTorch
2021TIPSparse gradient regularized deep retinex network for robust low-light image enhancement paperRetinex-NetPyTorch
2021TCSVTLow-light image enhancement via progressive-recursive network paperPRIENPyTorch
2021TIPBand representation-based semi-supervised low-light image enhancement: Bridging the gap between signal fidelity and perceptual quality paperDRBNPyTorch
2021ICCVAdaptive unfolding total variation network for low-light image enhancement paperUTVNetCodePyTorch
2021ICCVSeeing dynamic scene in the dark: A high-quality video dataset with mechatronic alignment paperWang et al.CodePyTorch
2022CVPRSNR-aware Low-Light Image Enhancement paperSNR-AwareCodePyTorch
2022CVPRToward Fast, Flexible, and Robust Low-Light Image Enhancement paperSCICodePyTorch
2022CVPRURetinex-Net: Retinex-Based Deep Unfolding Network for Low-Light Image Enhancement paperURetinex-NetCodePyTorch

📋Datasets

AbbreviationNumberFormatReal/SyneticVideoPaired/Unpaired/ApplicationDataset
LOL paper500RGBRealNoPairedDataset
SCIE paper4413RGBRealNoPairedDataset
MIT-Adobe FiveK paper5000RawRealNoPairedDataset
SID paper5094RawRealNoPairedDataset
DRV paper202RawRealYesPairedDataset
SMOID paper179RawRealYesPairedDataset
SDSD paper150RGBRealYesPairedDataset
LIME paper10RGBRealNoUnpairedDataset
NPE paper84RGBRealNoUnpairedDataset
MEF paper17RGBRealNoUnpairedDataset
DICM paper64RGBRealNoUnpairedDataset
VV24RGBRealNoUnpairedDataset
ExDARK paper7363RGBRealNoApplication (Object Detection)Dataset
BBD-100K paper10,000RGBRealYesApplication (Driving with diverse kinds of annotations)Dataset
DARK FACE paper6000RGBRealNoApplication (Face Recognition)Dataset
NightCity paper4297RGBRealNoApplication (Semantic Segmentation)Dataset

📋Metrics

AbbreviationFull-/Non-ReferencePlatformCode
MAE (Mean Absolute Error)Full-Reference
MSE (Mean Square Error)Full-Reference
PSNR (Peak Signal-to-Noise Ratio)Full-Reference
SSIM (Structural Similarity Index Measurement)Full-ReferenceMATLABCode
LPIPS (Learned Perceptual Image Patch Similarity)Full-ReferencePyTorchCode
LOE (Lightness Order Error)Non-ReferenceMATLABCode
NIQE (Naturalness Image Quality Evaluator)Non-ReferenceMATLABCode
PI (Perceptual Index)Non-ReferenceMATLABCode
SPAQ (Smartphone Photography Attribute and Quality)Non-ReferencePyTorchCode
NIMA (Neural Image Assessment)Non-ReferencePyTorch/TensorFlowCode/Code
MUSIQ (Multi-scale Image Quality Transformer)Non-ReferenceTensorFlowCode

📜</g-emoji>License

The code, platform, and dataset are made available for academic research purpose only.

📚</g-emoji>Citation

If you find the repository helpful in your resarch, please cite the following paper.

@article{LoLi,
  title={Low-Light Image and Video Enhancement Using Deep Learning: A Survey},
  author={Li, Chongyi and Guo, Chunle and Han, Linghao and Jiang, Jun and Cheng, Ming-Ming and Gu, Jinwei and Loy, Chen Change},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2021}
}

📋Paper

Official Version

arXiv Version

Chinese Version

📭Contact

lichongyi25@gmail.com; guochunle@nankai.edu.cn