Home

Awesome

NTIRE 2023 Video Colorization Challenge @ CVPR 2023

Track 1: Fréchet Inception Distance (FID) Optimization

google colab logo OpenXLab visitorsGitHub Stars

Please visit test_NTIRE23_Track_1_FID.py to evaluate our model.

We provide the colorized images HERE, and the reference images used to obtain the results HERE.

News

:briefcase: Dependencies and Installation

Environment configuration:

cd BiSTNet-NTIRE2023

# create a new anaconda env
conda create -n bistnet python=3.6
conda activate bistnet

# install pytortch
conda install pytorch==1.10.1 torchvision==0.11.2 torchaudio==0.10.1 cudatoolkit=11.3 -c pytorch -c conda-forge

# mmcv install (1.x, please do not use 2.x)
pip install -U openmim
mim install mmcv-full

# install mmediting (0.x, please do not use 1.x)
git clone https://github.com/open-mmlab/mmediting.git
cd mmediting
pip3 install -e .

# install other pip pkgs 
cd .. && pip install -r pip_requirements.txt

:gift: Checkpoints

NameURLScriptFIDCDC
BiSTNetmodeltest_NTIRE23_Track_1_FID.py21.53720.001717

:zap: Quick Inference for NTIRE 2023 Video Colorization Challenge

This version is specifically designed for NTIRE 2023 Video Colorization Challenge @ CVPR 2023. We colorize every 50 frames with two exemplars, see clip 001 for an example.

conda activate bistnet && cd BiSTNet-NTIRE2023
CUDA_VISIBLE_DEVICES=0 python test_NTIRE23_Track_1_FID.py

For more details please refer to test_NTIRE23_Track_1_FID.py.

:zap: Quick Inference for BiSTNet version

This version allows arbitrary number of input video frames with two exemplars. See clip fanghua234 for an example.

conda activate bistnet && cd BiSTNet-NTIRE2023
CUDA_VISIBLE_DEVICES=0 python test_BiSTNet.py

For more details please refer to test_BiSTNet.py.

License

BiSTNet is released under the MIT license, while some methods adopted in this project are with other licenses. Please refer to LICENSES.md for the careful check, if you are using our code for commercial matters. Thank @milmor so much for bringing up this concern about license.

Citation

If this work is helpful for your research, please consider citing the following entry.

@article{bistnet,
  title={BiSTNet: Semantic Image Prior Guided Bidirectional Temporal Feature Fusion for Deep Exemplar-based Video Colorization},
  author={Yang, Yixin and Peng, Zhongzheng and Du, Xiaoyu and Tao, Zhulin and Tang, Jinhui and Pan, Jinshan},
  journal={arXiv preprint arXiv:2212.02268},
  year={2022}
}

Acknowledgement

Part of our codes are taken from DeepExemplar, RAFT, HED and ProtoSeg. Thanks for their awesome works.