Awesome
Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation
<a href="https://arxiv.org/abs/2303.07937"><img src="https://img.shields.io/badge/arXiv-2303.07937-%23B31B1B"></a> <a href="https://ku-cvlab.github.io/3DFuse/"><img src="https://img.shields.io/badge/Project%20Page-online-brightgreen"></a> <br>
<p align="center"> <img src="imgs/1.gif" width="40%"> <img src="imgs/2.gif" width="40%"> <img src="imgs/3.gif" width="40%"> <img src="imgs/4.gif" width="40%"> </p> This is official implementation of the paper "Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation". The last column of each example is our result.⚡️News
❗️2023.04.10: We've opened the HuggingFace Demo! Also, we fixed minor issues, including the seed not being fixed.
❗️2023.03.31: We found that we typed an incorrect version of the model for point cloud inference. The fixed commit produces much better results.
Introduction
<center> <img src="https://ku-cvlab.github.io/3DFuse/imgs/3dfuse.png" width="100%" height="100%"> </center>We introduce 3DFuse, a novel framework that incorporates 3D awareness into pretrained 2D diffusion models, enhancing the robustness and 3D consistency of score distillation-based methods. For more details, please visit our project page!
🔥TODO
- 3D Generation/Gradio Demo Code
- HuggingFace🤗 Demo Release
- Colab Demo Release
- Mesh Converting Code
Installation
Please follow installation.
Interactive Gradio App
for Text-to-3D / Image-to-3D
Enter your own prompt and enjoy! With this gradio app, you can preview the point cloud before 3D generation and determine the desired shape.
python gradio_app.py
# or python gradio_app.py --share
<img src='imgs/ex1.png'>
<img src='imgs/ex2.png'>
Text-to-3D Generation
After modifying the run.sh
file with the desired prompt and hyperparameters, please execute the following command:
sh run.sh
Acknowledgement
We would like to acknowledge the contributions of public projects, including SJC and ControlNet whose code has been utilized in this repository.
Citation
@article{seo2023let,
title={Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation},
author={Seo, Junyoung and Jang, Wooseok and Kwak, Min-Seop and Ko, Jaehoon and Kim, Hyeonsu and Kim, Junho and Kim, Jin-Hwa and Lee, Jiyoung and Kim, Seungryong},
journal={arXiv preprint arXiv:2303.07937},
year={2023}
}