Awesome
ViTMatte for Nuke
Introduction
This project brings ViTMatte - Boosting Image Matting with Pretrained Plain Vision Transformers to The Foundry's Nuke.
ViTMatte is a natural matting neural network that can pull high-quality alphas from garbage mattes (trimaps).
This implementation wraps ViTMatte into a single Inference node in Nuke, allowing it to be easily installed on any Nuke 13+ system running Linux or Windows.
While ViTMatte works best on still images and doesn't have temporal stability, it can still be helpful for pulling difficult mattes, especially those with fine details like hair and fur.
<div align="center"> </div>https://github.com/rafaelperez/ViTMatte-for-Nuke/assets/1684365/bc02567a-5d95-4f5d-a8ca-45ed1e9b337c
Compatibility
Nuke 13.2+, tested on Linux and Windows.
Features
- High quality natural matting results
- Fast, less than one second per frame (2K).
- 4K Support even on 8GB GPUs.
- Commercial use license.
- Overscan and bounding box support
- Easy installation using Nuke's Cattery system
[!NOTE] Nuke 14 and later versions are recommended. Nuke 13 has significantly slower performance due to its PyTorch version.
Installation
- Download and unzip the latest release from here.
- Copy the extracted
Cattery
folder to.nuke
or your plugins path. - In the toolbar, choose Cattery > Update or simply restart Nuke.
ViTMatte will then be accessible under the toolbar at Cattery > Segmentation > ViTMatte.
⚠️ Extra Steps for Nuke 13
- Add the path for ViTMatte to your
init.py
:
import nuke
nuke.pluginAddPath('./Cattery/vitmatte')
- Add an menu item to the toolbar in your
menu.py
:
import nuke
toolbar = nuke.menu("Nodes")
toolbar.addCommand('Cattery/Segmentation/ViTMatte', 'nuke.createNode("vitmatte")', icon="vitmatte.png")
Quick Start
ViTMatte creates high-quality mattes from garbage mattes in just a few clicks, using a trimap to identify edges and semi-transparent areas.
What is a Trimap?
A trimap is a grayscale image that helps ViTMatte know what to focus on. Black means transparent, white means opaque, and gray means "needs work".
-
To get started, connect an image with an alpha channel to the ViTMatte node.
-
Use the Edge Thickness control to fine-tune the thickness of the trimap.
-
Adjust the Detail level to refine the matte.
-
If needed, disable Fast mode to further enhance the matte quality.
[!NOTE] For a quick and easy way to create garbage mattes, check out the SegmentAnything for Nuke tool.
In Depth Tutorial
Comp supervisor Alex Villabon made an in-depth tutorial and demo about VITMatte on his YouTube channel.
Check out the video and his channel for more awesome Nuke tips. Thanks, Alex!
Release Notes
Latest version: 1.2
- Memory Optimization: Reduced peak memory usage by 3x, enabling 4K image processing on 8GB GPUs.
- Improved Gizmos Trimap: Swapped
Dilate
forErode
to reduce boiling edges.
Version 1.1
- Added fast model. This model is more suited for improving binary masks, like those from Segment Anything for Nuke.
- Reduced memory usage with fp16 quantization
- Support to Nuke 13
- Improved Gizmo interface
- Added overscan support
- Added bounding box support
- Fixed padding inference issues
- Easy installation with Cattery package
- New toolbar icon! ✨
License and Acknowledgments
ViTMatte.cat is licensed under the MIT License, and is derived from https://github.com/hustvl/ViTMatte.
While the MIT License permits commercial use of ViTMatte, the dataset used for its training may be under a non-commercial license.
This license does not cover the underlying pre-trained model, associated training data, and dependencies, which may be subject to further usage restrictions.
Consult https://github.com/hustvl/ViTMatte for more information on associated licensing terms.
Users are solely responsible for ensuring that the underlying model, training data, and dependencies align with their intended usage of RIFE.cat.
Citation
@article{yao2024vitmatte,
title={ViTMatte: Boosting image matting with pre-trained plain vision transformers},
author={Yao, Jingfeng and Wang, Xinggang and Yang, Shusheng and Wang, Baoyuan},
journal={Information Fusion},
volume={103},
pages={102091},
year={2024},
publisher={Elsevier}
}