Home

Awesome

FreeControl: Training-Free Spatial Control of Any Text-to-Image Diffusion Model with Any Condition

[Paper] [Project Page] <br>

Sicheng Mo<sup>1*</sup>, Fangzhou Mu<sup>2*</sup>, Kuan Heng Lin<sup>1</sup>, Yanli Liu<sup>3</sup>, Bochen Guan<sup>3</sup>, Yin Li<sup>2</sup>, Bolei Zhou<sup>1</sup> <br> <sup>1</sup> UCLA, <sup>2</sup> University of Wisconsin-Madison, <sup>3</sup> Innopeak Technology, Inc <br> <sup>*</sup> Equal contribution <br> Computer Vision and Pattern Recognition (CVPR), 2024 <br>

<p align="center"> <img src="docs/assets/teaser1.jpg" alt="teaser" width="90%" height="90%"> </p>

Overview

This is the official implementation of FreeControl, a Generative AI algorithm for controllable text-to-image generation using pre-trained Diffusion Models.

Changelog

Getting Started

Environment Setup

conda env create -f environment.yml
conda activate freecontrol
pip install -U diffusers 
pip install -U gradio

Sample Semantic Bases

Gradio demo

python gradio_app.py

Galley:

We are building a gallery of images generated with FreeControl. You are welcome to share your generated images with us.

Contact

Sicheng Mo (smo3@cs.ucla.edu)

Reference

@article{mo2023freecontrol,
  title={FreeControl: Training-Free Spatial Control of Any Text-to-Image Diffusion Model with Any Condition},
  author={Mo, Sicheng and Mu, Fangzhou and Lin, Kuan Heng and Liu, Yanli and Guan, Bochen and Li, Yin and Zhou, Bolei},
  journal={arXiv preprint arXiv:2312.07536},
  year={2023}
}