Awesome
This is an implementation of MotionCtrl for ComfyUI
MotionCtrl: A Unified and Flexible Motion Controller for Video Generation
Install
-
Clone this repo into custom_nodes directory of ComfyUI location
-
Run pip install -r requirements.txt
-
Download the weights of MotionCtrl motionctrl.pth and put it to
ComfyUI/models/checkpoints
Nodes
Four nodes Load Motionctrl Checkpoint
& Motionctrl Cond
& Motionctrl Sample Simple
& Load Motion Camera Preset
& Load Motion Traj Preset
& Select Image Indices
&Motionctrl Sample
Tools
Motion Traj Tool Generate motion trajectories
<img src="assets/traj.png" raw=true>Motion Camera Tool Generate motion camera points
<img src="assets/camera.png" raw=true>Examples
base workflow
<img src="assets/base_wf.png" raw=true>https://github.com/chaojie/ComfyUI-MotionCtrl/blob/main/workflow_motionctrl_base.json
<video controls autoplay="true"> <source src="assets/dog.mp4" type="video/mp4" /> </video>unofficial implementation "MotionCtrl deployed on AnimateDiff" workflow:
<img src="assets/scribble_wf.png" raw=true>https://github.com/chaojie/ComfyUI-MotionCtrl/blob/main/workflow_motionctrl.json
- Generate LVDM/VideoCrafter Video
- Select Images->Scribble
- Use AnimateDiff Scribble SparseCtrl