Awesome

ObjCtrl-2.5D: Training-free Object Control with Camera Pose

<div align='center'> <a href='https://wzhouxiff.github.io/'>Zhouxia Wang</a> &ensp; <a href='https://nirvanalan.github.io/'>Yushi Lan</a> &ensp; <a href='https://shangchenzhou.com/'>Shangchen Zhou</a> &ensp; <a href='https://www.mmlab-ntu.com/person/ccloy/index.html'>Chen Change Loy</a> </div> <div align='center'> <a href='https://www.mmlab-ntu.com/'>S-Lab, NTU Singapore</a> </div> <br>

</div>

This repo is an official implementation of ObjCtrl-2.5D: Training-free Object Control with Camera Pose. ObjCtrl-2.5D models a 3D trajectory, extended from a 2D trajectory with depth information, as a sequence of camera poses, enabling object motion control using an existing camera motion control I2V generation model without training. Dynamic results achieved by our ObjCtrl-2.5D are provided in our .

📝 Changelog

202412: Release both code and demo (🤗 ) of ObjCtrl-2.5D.

⚙️ Environment

conda create -n objctrl2.5d python=3.10
conda activate objctrl2.5d
pip install -r requirements.txt

💫 Running Gradio Demo

python -m app

:books: Citation

If you make use of our work, please cite our paper.

@inproceedings{objctrl2.5d,
  title={{ObjCtrl-2.5D}: Training-free Object Control with Camera Poses},
  author={Wang, Zhouxia and Lan, Yushi and Zhou, Shangchen and Loy, Chen Change},
  booktitle={arXiv preprint arXiv:2412.07721},
  year={2024}
}

🤗 Acknowledgment

We appreciate the authors of SVD, CameraCtrl, SAM2, and ZoeDepth for their awesome works.

❓ Contact

For any question, feel free to email zhouzi1212@gmail.com.