Awesome
This is an implementation of ComfyUI MotionCtrl for SVD
Install
-
Clone this repo into custom_nodes directory of ComfyUI location
-
Run pip install -r requirements.txt
-
Download the weights of MotionCtrl for SVD motionctrl_svd.ckpt and put it to
ComfyUI/models/checkpoints
Examples
base workflow
<img src="assets/base_wf.png" raw=true>https://github.com/chaojie/ComfyUI-MotionCtrl-SVD/blob/main/workflow.json
A little exploration workflow: for videos with relatively static camera angles, applying the same MotionCtrl to each frame, then combining images from corresponding positions to create a new video. The nth frame image is taken from the generated nth frame and merged again.
https://github.com/chaojie/ComfyUI-MotionCtrl-SVD/blob/main/workflow_video.json
original video: https://github.com/chaojie/ComfyUI-MotionCtrl-SVD/blob/main/assets/original.mp4 generate video: https://github.com/chaojie/ComfyUI-MotionCtrl-SVD/blob/main/assets/svd.mp4