Home

Awesome

Deep_Video_Inpainting

Official pytorch implementation for "Deep Video Inpainting" (CVPR 2019, TPAMI 2020)
Dahun Kim*, Sanghyun Woo*, Joon-Young Lee, and In So Kweon. (*: equal contribution)
[Paper] [Project page] [Video results]

If you are also interested in video caption removal, please check [Paper] [Project page]

Update

This is recently tested under <b>Python=3.7, Pytorch=1.4, Cudatoolkit=10.0</b>.

<img src="./image/swing4.jpg" width="700" height="350"> <img src="./image/roller4.jpg" width="700" height="350">

Disclaimer

This is tested under <s>Python=3.6, PyTorch=0.4.0</s>
Python=3.7, Pytorch=1.4, Cudatoolkit=10.0.

Testing

  1. Download the trained weight 'save_agg_rec_512.pth' and place it in "./results/vinet_agg_rec/"
    Google drive: [weight-512x512] [weight-256x256]

  2. Create conda env and install Pytorch library.

conda create -n vinet python=3.7 -y
conda activate vinet
conda install pytorch=1.4 torchvision cudatoolkit=10.0 -c pytorch -y
  1. Compile Resample2d, Correlation dependencies.
bash ./install.sh
  1. Run the demo (the results are saved in "./results/vinet_agg_rec/davis_512/").
python demo_vi.py
  1. Optional: Run the video retargeting (Section 4.5)
python demo_retarget.py
<img src="./image/bmx-trees.png" width="700" height="140">

Citation

If you find the codes useful in your research, please cite:

@inproceedings{kim2019deep,
  title={Deep Video Inpainting},
  author={Kim, Dahun and Woo, Sanghyun and Lee, Joon-Young and Kweon, In So},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  pages={5792--5801},
  year={2019},}

@ARTICLE{kim2020vipami,
  author={Kim, Dahun and Woo, Sanghyun and Lee, Joon-Young and Kweon, In So},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  title={Recurrent Temporal Aggregation Framework for Deep Video Inpainting},
  year={2020},
  volume={42},
  number={5},
  pages={1038-1052},}