Home

Awesome

HDRTVNet++ [Paper Link]

Towards Efficient SDRTV-to-HDRTV by Learning from Image Formation

Xiangyu Chen*, Zheyuan Li*, Zhengwen Zhang, Jimmy S. Ren, Yihao Liu, Jingwen He,

Yu Qiao, Jiantao Zhou, Chao Dong

(* indicates equal contribution)

:star:The conference version is accepted to ICCV 2021.
:rocket:The conference repository can be found here.

Overview

Simplified SDRTV/HDRTV formation pipeline:

<img src="https://github.com/xiaom233/HDRTVNet-plus/blob/main/fig/Fig1(a)_content_foramtion_pipeline.png" width="600"/>

Overview of the method:

<img src="https://github.com/xiaom233/HDRTVNet-plus/blob/main/fig/Fig3_network_arch.png" width="900"/>

Getting Started

  1. Dataset
  2. Configuration
  3. How to test
  4. How to train
  5. Metrics
  6. Visualization

Dataset

We conduct a dataset using videos with 4K resolutions under HDR10 standard (10-bit, Rec.2020, PQ) and their counterpart SDR versions from Youtube. The dataset consists of a training set with 1235 image pairs and a test set with 117 image pairs. Please refer to the paper for the details on the processing of the dataset. The dataset can be downloaded from Baidu Netdisk (access code: 6qvu) or OneDrive (access code: HDRTVNet). The training set is uploaded after subsection compression since it's too large. Please download the complete dataset to unzip.

We also provide the original Youtube links of these videos, which can be found in this file. Note that we cannot provide the download links since we do not have the copyright to distribute. Please download this dataset only for academic use.

Configuration

Please refer to the requirements. Matlab is also used to process the data, but it is not necessary and can be replaced by OpenCV.

How to test

We provide the pretrained models to test, which can be found in ./pretrained_models. Since our method is casaded of three steps, the results also need to be inferenced step by step.

cd codes
python test.py -opt options/test/test_AGCM.yml
python test.py -opt options/test/test_LE.yml
python test.py -opt options/test/test_Ensemble_AGCM_LE.yml
python test.py -opt options/test/test_HR.yml

How to train

cd codes
python train.py -opt options/train/train_AGCM.yml
python train.py -opt options/train/train_LE.yml
python train.py -opt options/train/train_Ensemble_AGCM_LE.yml
python train.py -opt options/train/train_HR.yml

Metrics

Five metrics are used to evaluate the quantitative performance of different methods, including PSNR, SSIM, SR_SIM, Delta E<sub>ITP</sub> (ITU Rec.2124) and HDR-VDP3. Since the latter three metrics are not very common in recent papers, we provide some reference codes in ./metrics for convenient usage.

Visualization

Since HDR10 is an HDR standard using PQ transfer function for the video, the correct way to visualize the results is to synthesize the image results into a video format and display it on the HDR monitor or TVs that support HDR. The HDR images in our dataset are generated by directly extracting frames from the original HDR10 videos, thus these images consisting of PQ values look relatively dark compared to their true appearances. We provide the reference commands of our extracting frames and synthesizing videos in ./scripts. Please use MediaInfo to check the format and the encoding information of synthesized videos before visualization. If circumstances permit, we strongly recommend to observe the HDR results and the original HDR resources by this way on the HDR dispalyer.

If the HDR displayer is not available, some media players with HDR render can play the HDR video and show a relatively realistic look, such as Potplayer. Note that this is only an approximate alternative, and it still cannot fully restore the appearance of HDR content on HDR monitors.

Citation

If our work is helpful to you, please cite our paper:

@article{chen2023towards,
  title={Towards Efficient SDRTV-to-HDRTV by Learning from Image Formation},
  author={Chen, Xiangyu and Li, Zheyuan and Zhang, Zhengwen and Ren, Jimmy S and Liu, Yihao and He, Jingwen and Qiao, Yu and Zhou, Jiantao and Dong, Chao},
  journal={arXiv preprint arXiv:2309.04084},
  year={2023}
}

The conference version:

@InProceedings{chen2021hdrtvnet,
    author    = {Chen, Xiangyu and Zhang, Zhengwen and Ren, Jimmy S. and Tian, Lynhoo and Qiao, Yu and Dong, Chao},
    title     = {A New Journey From SDRTV to HDRTV},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2021},
    pages     = {4500-4509}
}

Acknowledgment

The code is inspired by BasicSR.