Home

Awesome

FHDR: HDR Image Reconstruction from a Single LDR Image using Feedback Network

arXiv

This repository contains the code for our FHDR work accepted at GlobalSIP.

<p align="center"> <img src="https://user-images.githubusercontent.com/24846546/166098107-5d9d394e-6e32-4963-9417-7c1d152d717d.png" width="70%"> </p>

Table of contents:

Abstract

High dynamic range (HDR) image generation from a single exposure low dynamic range (LDR) image has been made possible due to the recent advances in Deep Learning. Various feed-forward Convolutional Neural Networks (CNNs) have been proposed for learning LDR to HDR representations. <br><br> To better utilize the power of CNNs, we exploit the idea of feedback, where the initial low level features are guided by the high level features using a hidden state of a Recurrent Neural Network. Unlike a single forward pass in a conventional feed-forward network, the reconstruction from LDR to HDR in a feedback network is learned over multiple iterations. This enables us to create a coarse-to-fine representation, leading to an improved reconstruction at every iteration. Various advantages over standard feed-forward networks include early reconstruction ability and better reconstruction quality with fewer network parameters. We design a dense feedback block and propose an end-to-end feedback network- FHDR for HDR image generation from a single exposure LDR image. Qualitative and quantitative evaluations show the superiority of our approach over the state of-the-art methods.

Setup

Pre-requisites

requirements.txt has been provided for installing Python dependencies.

pip install -r requirements.txt

Dataset

The dataset is to comprise of LDR (input) and HDR (ground truth) image pairs. The network is trained to learn the mapping from LDR images to their corresponding HDR ground truth counterparts.

The dataset should follow the following folder structure -

> dataset

    > train

        > LDR

            > ldr_image_1.jpg/png
            > ldr_image_2.jpg/png
            .
            .

        > HDR

            > hdr_image_1.hdr/exr
            > hdr_image_2.hdr/exr
            .
            .

    > test

Note: The pre-trained models were trained on 256x256 size images.

Training

After the dataset has been prepared, the model can be trained using the train.py file.

python3 train.py

The corresponding parameters/options for training have been specified in the options.py file and can be easily altered. They can be logged using -

python3 train.py --help

Pretrained models

Pre-trained models can be downloaded from the below-mentioned links.

These models have been trained with the default options, on 256x256 size images for 200 epochs, in accordance with the paper.

Here is a graph plotting the performance vs iteration count.

<img src="https://user-images.githubusercontent.com/24846546/71311250-ed28ab00-2415-11ea-9842-f84b5999161e.png" width="40%">

Evaluation

The performance of the network can be evaluated using the test.py file -

python3 test.py --ckpt_path /path/to/pth/checkpoint
python3 test.py --log_scores

Citation

If you use this code for your research, please cite the following paper.

@INPROCEEDINGS{8969167,
    author={Z. {Khan} and M. {Khanna} and S. {Raman}},
    booktitle={2019 IEEE Global Conference on Signal and Information Processing (GlobalSIP)},
    title={FHDR: HDR Image Reconstruction from a Single LDR Image using Feedback Network},
    year={2019},
    pages={1-5},
    doi={10.1109/GlobalSIP45357.2019.8969167}
}

Acknowledgement

This research was supported by the Science and Engineering Research Board (SERB) Core Research Grant.