Home

Awesome

Vision-Infused Deep Audio Inpainting

We present a vision-infused method that can deal with both audio-only and audio-visual associated inpainting Inspired by image inpainting, called Vision-Infused Audio Inpainter (VIAI).

[Project] [Paper] [Demo]

<img src='./misc/pipeline2.png' width=880>

Requirements

Dataset

The MUSICES dataset can be accessed here.

<img src='./misc/datastatistic.png' width=880>

Training and Testing

We are still sorting out the code. For now it is not complete thus not runable, but the architecture is revealed. Please wait for more details.

License and Citation

The use of this software is RESTRICTED to non-commercial research and educational purposes.

@InProceedings{Zhou_2019_ICCV,
  author = {Zhou, Hang and Liu, Ziwei and Xu, Xudong and Luo, Ping and Wang, Xiaogang},
  title = {Vision-Infused Deep Audio Inpainting},
  booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
  month = {October},
  year = {2019}
}

Acknowledgement

The structure of this codebase is borrowed from pix2pix and wavent_vocoder.