Awesome

[ECCV2024] DualDn: <br> Dual-domain Denoising via Differentiable ISP

👉 Project Page <br>

Ruikang Li, Yujin Wang, Shiqi Chen, Fan Zhang, Jinwei Gu and Tianfan Xue <br>

News

• Sept 29, 2024: Paper accepted at ECCV 2024. 😊
• Nov 4, 2024: Training and inferencing code released. 🌹
• Nov 10, 2024: Pre-trained models and visual results released. Feel free to download! 📝
• Nov 15, 2024: Project page has evolved to the next generation—check it out! 💢

TODO Lists

• Demo on Huggingface for processing real-captured images.
• We have captured a new test set with real_captured RAW images, and we will release a new benchmark for dual-denoising!
• Evaluate on SIDD benchmark.

Abstract: There are two typical ways to inject a denoiser into the Image Signal Processing (ISP) pipeline: applying a denoiser directly to captured raw frames (raw domain) or to the ISP's output sRGB images (sRGB domain). However, both approaches have their limitations. Residual noise from raw-domain denoising can be amplified by the subsequent ISP processing, and the sRGB domain struggles to handle spatially varying noise since it only sees noise distorted by the ISP. Consequently, most raw or sRGB domain denoising works only for specific noise distributions and ISP configurations. Unlike previous single-domain denoising, DualDn consists of two denoising networks: one in the raw domain and one in the sRGB domain. The raw domain denoising adapts to sensor-specific noise as well as spatially varying noise levels, while the sRGB domain denoising adapts to ISP variations and removes residual noise amplified by the ISP. Both denoising networks are connected with a differentiable ISP, which is trained end-to-end and discarded during the inference stage.

Network Architecture

Installation

See INSTALL.md for the installation of environment and dependencies required to run DualDn.

Demo

Train and Evaluation

We trained DualDn on a single GPU ONLY using: (1) clean raw images, (2) a reasonable noise model.

We chose the MIT-Adobe FiveK Dataset for training, as it's a robust dataset containing multiple raw images in DNG format with EXIF metadata. Although some images contain noise, MIT-Adobe FiveK is sufficient for training DualDn to generalize effectively to in-the-wild scenes.

And we believe that if more clean raws or a more accurate noise model are given, DualDn’s performance could improve even further.

HINTS: We only provide models and results trained using Restormer backbone, as it demonstrated the best performance in DualDn. However, you can also train DualDn with other backbones, such as SwinIR or MIRNet-v2, by following our instructions. Reference evaluation metrics are also available in our paper.

Results

Citation

@inproceedings{li2025dualdn,
  title={Dualdn: Dual-domain denoising via differentiable isp},
  author={Li, Ruikang and Wang, Yujin and Chen, Shiqi and Zhang, Fan and Gu, Jinwei and Xue, Tianfan},
  booktitle={European Conference on Computer Vision},
  pages={160--177},
  year={2025},
  organization={Springer}
}

License

<a rel="license" href="https://creativecommons.org/licenses/by-nc/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://licensebuttons.net/l/by-nc/4.0/88x31.png" />

This repository is licensed under Attribution-NonCommercial 4.0 International

1. Attribution — Proper credit must be given, including a link to the license and an indication of any modifications made. This should be done in a reasonable manner, without implying endorsement by the licensor

2. NonCommercial — The Algorithm may NOT be used for commercial purposes. This includes, but is not limited to, the sale, licensing, or integration of the Software into commercial products or services.

For collaboration or inquiries, please contact us.

Acknowledgment

This code is based on the BasicSR toolbox.